From:	CRDGW2::CRDGW2::MRGATE::"SMTP::CRVAX.SRI.COM::RELAY-INFO-VAX"  8-FEB-1992 13:24:31.70
To:	ARISIA::EVERHART
CC:	
Subj:	Symbol Subst. Summary (was "Weird DCL Behavior")

From:	RELAY-INFO-VAX@CRVAX.SRI.COM@SMTP@CRDGW2
To:	Everhart@Arisia@MRGATE

Received:  by crdgw1.ge.com (5.57/GE 1.122)
	 id AA26025; Sat, 8 Feb 92 12:55:10 EST
Received: From UCBVAX.BERKELEY.EDU by CRVAX.SRI.COM with TCP; Sat,  8 FEB 92 09:48:12 PST
Received: by ucbvax.Berkeley.EDU (5.63/1.43)
	id AA06450; Sat, 8 Feb 92 09:38:35 -0800
Received: from USENET by ucbvax.Berkeley.EDU with netnews
	for info-vax@kl.sri.com (info-vax@kl.sri.com)
	(contact usenet@ucbvax.Berkeley.EDU if you have questions)
Date: 2 Feb 92 23:30:06 GMT
From: att!oucsboss!oucsace!afeldman@ucbvax.Berkeley.EDU  (Alan E. Feldman)
Organization: Ohio University CS Dept,. Athens
Subject: Symbol Subst. Summary (was "Weird DCL Behavior")
Message-Id: <4851@oucsace.cs.OHIOU.EDU>
Sender: info-vax-request@kl.sri.com
To: info-vax@kl.sri.com

Here at long last is the Weird DCL Behavior Summary. In it 
I show that the "The & just translates first." explanation does
not work for all cases.

The original problem was as follows:

If you run the following:

$  ZERO = "ONE" 
$  ONE  = "TWO"        
$  TWO  = "THREE" 
$  THREE = "FOUR"

, then:

Commands:        -->   SYS$OUTPUT:

$! DE = '&ZERO'        $! DE = 'ONE'
$! DE = ''&ZERO'       $! DE = ''ONE'
$! DE = "''ZERO'"      $! DE = "'ZERO'"
$! DE = "''&ZERO'"     $! DE = "'ONE'"
$! DE = "&''ZERO'"     $! DE = "&'ZERO'"

Bizarre, eh?


$! DE := ''&ZERO'      $! DE := ''ONE'
$  DE := ''&ZERO'      $  DE := &ZERO
$  SHOW SYMBOL DE      $  SHOW SYMBOL DE
                         DE = "&ZERO"

Here, ZERO is translated only in the commented line. (Of course it
shouldn't be translated in any of the above three lines since '' should
just disappear if there is any translation at all.) 

Also:

$! (1):
$  DE := '&ZERO'        $  DE := TWO
$  SHOW SYMBOL DE       $  SHOW SYMBOL DE
                          DE = "TWO"

$! (2):
$  DE := &'ZERO'        $  DE := &ONE
$  SHOW SYMBOL DE       $  SHOW SYMBOL DE
                          DE = "&ONE"

$! (3):
$  DE  = '&ZERO'        $  DE  = TWO
$  SHOW SYMBOL DE       $  SHOW SYMBOL DE
                          DE = "THREE"

$! (4):
$  DE  = &'ZERO'        $  DE  = &ONE
$  SHOW SYMBOL DE       $  SHOW SYMBOL DE
                          DE = "THREE"

Cases 2 and 4 behave pretty much as the documentation indicates.
However, I claimed that the odd cases do not behave as the documentation
indicates. I will explain this in the section titled "__ The '&
operator."  Also, in that section I will show that the "& just 
translates first" explanation does not work for all cases.

But now, the undisputed weirdity first. 

(NOTE: In this article, tick means apostrophe.) 

__ Symbol Substitution Within Commented Lines

As shown above, the commented lines are reported strangely to
SYS$OUTPUT. Sometimes two ticks magically become one, without any symbol
translation. Also, ampersands translate if preceded by a tick. No one
could fully explain why this happens. Further, as already posted before,
Frank Monroe (VAXS06@v7.vitro.com) noted that the bizarre translation is
a function of the cases of the letters in the symbol name. Using
variations with zERO instead of ZERO, sometimes zERO disappears
altogether! 

Witness:

Commands:            SYS$OUTPUT:

$  zERO = "ONE"      $  zERO = "ONE"

$! DE = '&zERO'      $! DE = ''
$! DE = ''&zERO'     $! DE = '''
$! DE = "''zERO'"    $! DE = "'zERO'"
$! DE = "''&zERO'"   $! DE = "''"
$! DE = "&''zERO'"   $! DE = "&'zERO'"
$! DE = "''&zERO"    $! DE = "'

Also, Frank Monroe <VAXS06@V7.VITRO.COM> thought that the symbol
substitution for '& was "normal," but admitted that the documentation
doesn't explain it correctly and/or clearly. (I never meant to claim
that there was a bug. I meant to claim that the '& operator is an
undocumented feature.) He fully agreed that the translation that occurs
within commented lines is bizarre. In reference to this he wrote in two
separate mail messages the following: 

> DEC says that the substitution in the comment lines is intended
> behavior.  They will, however, have the documentation updated to reflect
> this behavior. 

> I asked DEC, since this is intended behavior, to shed some light on why
> the developers decided to do this so I can better understand what is
> going on here. And here is the response I got:  "I'm sorry but I don't
> have that information. If you believe that the behaviour isn't correct I
> can log this as a bug with the DCL maintainers.  From what I can see it
> appears intentional due to the way DCL parses command lines, is not due
> to any intentional requirements.  Let me know whether you want this
> logged as a problem with DCL." 


Another enlightening response came from Mike
Moroney@ramblr.enet.dec.com. He says: 

> A little history:  Back in the dark ages (VMS V1 or so) the ' character was
> interpreted immediately, regardless of whether the line was commented or
> not (in other words ' was interpreted before ! ).  It was decided that this
> was incorrect since many people use ' as a quote in comments so this
> behavior messed up comments.  However, many other people learned the 'hack'
> of starting a command procedure with $! 'f$verify(0) which turned off
> echoing before any of the command procedure was echoed if echoing was on.
> DCL was changed to ignore ' in comments except for the special case of
> 'f$verify().  Of course the comments are really interpreted to do this
> but the extent is determined [by] whether it's a commented line or not.
> So ' *is* interpreted in comments, but only in looking for and processing
> f$verify().  I guess it's forcing the interpretation of & in looking for
> the f$verify, and is perhaps a bug when the interpreted line is printed.

Well, I checked F$VERIFY in the HELP utility and it says:

Lexicals

  F$VERIFY

    Description

         The lexical function F$VERIFY returns an integer value indicating
         .
         .
         .
         DCL performs the F$VERIFY function even if it appears after a
         comment character, if it is enclosed in single quotation marks
         (' ').  This is the only processing that DCL performs within a
         comment.

NOT! (Only in reference to the last sentence of the HELP excerpt.) 


__ The '& operator:

Cases 1 and 3 involve what I call the '& operator.

Carl@SOL1.GPS.CALTECH.EDU wrote...

> The apostrophe says "we have to do a symbol substitution.["]  It then
> evaluates the symbol following the ampersand.  At this point, it's doing
> a symbol evaluation, which means that the ampersand gets parsed as an
> operator, so the symbol substitution for the ampersand occurs.  DCL
> apparently then exits from the symbol substitution algorithm without
> doing a second substituion on the comment lines.  On the non-comment
> lines, it then proceeds with its parsing and performs the second
> substitution.  As far as I know, symbol substitution isn't fully
> documented anywhere outside of DEC development groups. 

Mike Moroney@ramblr.enet.dec.com wrote...

> The command procedure behaves as I expect except when the sequence '& is
> encountered.  I would expect that, in the interpretation of ', either
> the & would be interpreted as the end of the symbol, or it would be
> interpreted as the first character of a symbol name.  In the first case,
> I'd expect the '&ZERO have the ' evaluate a null symbol (no characters
> between ' and &), leaving a null string, therefore the resultant string
> would be &ZERO, to be interpreted later. In the second case (which I
> considered unlikely) I'd expect the ' to try and interpret the symbol
> name &ZERO which would evaluate to the null string. It turns out that
> this is the case, with the exception that the & is interpreted at this
> time, which I didn't know ' did. 

Others at first said the '& operated normally. However, the
documentation disagrees with what really happens. After some
experimentation, I found that '&symbol_name' translates symbol_name
exactly twice during phase 1. If the "final" result of translating all
original occurrences of 'symbol_name and '&symbol_name in the command
string results in new occurrences of 'symbol_name and/or '&symbol_name,
then they translate, and so on, until no new occurrences of ' or '&
appear. Note that the first translation from a '& just gets translated.
For example, 

Example A:  '&ZERO' : ZERO --> ONE --> TWO . 

Example B:  But suppose instead that we had ZERO = "'ONE'". Then 

            '&ZERO' : ZERO --> 'ONE' --> null 

This is because while ONE is a symbol, 'ONE' isn't.

On the right side of an = this null yields

%DCL-W-EXPSYN, invalid syntax,.... 

Thus, '& has the same iterative properties that ' has if you consider
the entire sequence of Example A to be a single step. 

This is in contrast to &'ZERO'! Here, 'ZERO' translates in phase 1
giving &ONE. &ONE then translates in phase 2 only if it is not
surrounded by quotation marks and it is not on the right side of a :=
operator. 

(You can check which phase things happen in by running the command file
with verification set on. The results of phase 1 are reported instead of
the actual command strings. This is actually documented in the
description of the SET VERIFY command.) 

Now if the discussion of iterative translation from section 7.4.1 of the
DCL Concepts manual applied to the '& operator (as some respondents
suggested), then Example B wouldn't screw up. That's because that
discussion has a symbol enclosed in ticks after the first translation
and implies that these newly-appearing ticks perform the next
translation. With the '& opeartor, the newly-appearing ticks in Example
B actually screw it up! (It seems to me that if it did sort of work,
there would be an extra tick leftover that would cause trouble anyway.) 

In other words, Section 7.4.1 says that under the right conditions: If
THE RESULT of a translation PERFORMED BY TICKS is a symbol ENCLOSED IN
NEW TICKS, then the symbol enclosed in the new ticks is evaluated by
those new ticks. But that is not the case with '&ZERO'! There is no
translation (intermediate or otherwise) that results in 'ONE' which is
what would have to happen for the discussion to apply. While assuming 
that &ZERO goes to ONE first, ONE is then surrounded by old ticks and
can then be translated to TWO. However, this has nothing to do with
the point of section 7.4.1. That section NEVER mentions &'s.


Further,

the "& just translates first" explanation fails to properly explain
the results of Example B. In that example, if & just translated first,
then we would see instead:

'&ZERO' --> ''ONE'' --> ONE

The fact that this is *not* the observed behavior when '&ZERO' is 
processed by DCL shows that "& just translates first" is not the
correct explanation.


In section 7.4.1 we have the following:

$ MAC = "5"
$ A = "'MAC'"
$ B = 'A'
$ SHOW SYMBOL B
  B = "5"

But if you try

$ B = '&A'

you get the invalid syntax error beause '&A' yields null. This is
analagous to Example B above. In this example, when we are at the step
where it says $ B = 'A', A is translated to 'MAC'. This involves the
emergence of two NEW ticks which are actually part of the expression
that was equated to the symbol A. However, there are no new ticks that
emerge with my original examples of '&ZERO'. None of the first four
commands at the beginning of this article have any ticks as part of the
expressions. Therefore, this discussion does not apply. Furthermore,
ampersands are not mentioned anywhere in that section anyway! The whole
section deals *only* with the use of ticks. 

Some respondents said for both cases ( '& and &' ) the ' translates and
the & translates. That is an oversimplification as illustrated above.
The &' combination works as documented, but the behavior of the '&
"combination operator" violates the rules stated in DCL Concepts (V5)
Section 7.3. It clearly says there that the & operator works only in
phase 2. Also, you can try 

$ DE := &'&ZERO' 

$ DE = &'&ZERO'

and 

$ DE = "&''&ZERO'" 

as further illustrations.

Also, in section 7.2.2 of the DCL Concepts Manual for V5, I quote

> Observe the following rules:
> 
> * ...
> 
> * ...
> 
> * You cannot use ampersands to request substitution within character 
>   strings enclosed in quotation marks.

Yet the fact that DE = "''&ZERO'" yields DE = "TWO" is clearly a
violation of the above rule! 

Further, the whole purpose of the ampersand is to allow one to do the
follwing neat trick: 

$  P1 = "FRED.DAT"
$  COUNT = 1
$  TYPE &P'COUNT'

The result is

$  TYPE FRED.DAT

.


__ Missing closing tick oddity:

If the closing tick on a symbol is missing and there is a quote there
instead, e.g., "''symbol_name", then DCL gets confused and replaces
"''symbol_name" with ". Putting the closing tick in makes it work
properly. 

For example

$  DE = "''ZERO"

gets reported to SYS$OUTPUT as 

$  DE = "

.

There you have it! Thanks to all the respondents.

Disclaimer: The opinions herein are not necessarily those of anyone
other than the author, and are not sold in any store. 

Alan E. Feldman            &-)
Dept. of Physics
Ohio U., Athens, OH 45701

feldman@oualv1.phy.ohiou.edu, also feldman@umdenp.bitnet