From: MERC::"uunet!CRVAX.SRI.COM!RELAY-INFO-VAX" 5-SEP-1992 13:22 5-SEP-1992 13:22:00.00 To: INFO-VAX CC: Subj: Announcing VMS_SHARE version 8.0 Dear info-vax'ers, A major update to the VMS_SHARE package is now available from my file server (details below). I've collected various suggestions and bug reports over the last two years and most are included in this release. For those who don't know, VMS_SHARE allows files to be packaged in a way that allows them to be mailed through the networks unscathed, avoiding all the nasty corruptions that mailers can throw at them. Version 8.0 includes several new features and I've included the relevant portion of the release notes below to give you an overview of the changes. Further revised documentation is available with the package. You can fetch the package by sending a mail message to VMSSERV @ UK.AC.KCL.CC.OAK (from JANET) or VMSSERV%OAK.CC.KCL.AC.UK @ NSFNET-RELAY.AC.UK (from Internet) which contains the following line in the body of the message: SEND VMS_SHARE.PACKAGE Messages are processed in batches so please accept that there may be a delay before you receive the file (it's in 14 VMS_SHARE'd parts of 30 blocks each). If you have not received anything within 3 days or so, please drop me a line. Please let me know if you have any problems or difficulties in using it or unpacking it. With so many new things, there may be a few outstanding problems which my testing has not shown up. I'll try to reply individually to those who send bug reports but if I'm overwhelmed I'll send a single generic reply to info-vax. As always, keep the suggestions coming. Regards, Andy Harper Kings College London, England V M S _ S H A R E R E L E A S E N O T E S Version 8.0 Sep 1992 1. INTRODUCTION Version 8.0 of VMS_SHARE is a major release. It introduces some new features for users and fixes several bugs. 2. NEW FEATURES The following new features are included. 2.1 Parameters and Qualifiers on Command Line Most VMS_SHARE options and parameters can now be supplied directly on the command line. The logical names previously used to modify defaults are still accepted and behave as before but the use of a command line qualifier to set a value overrides that set by the logical name. New command line format: $ VMS_SHARE files sharefile [qualifiers] where qualifiers can be any or all of: /COMPRESS Turns on the run-length compression option. Should be used with caution as it can take considerably longer to encode the files. /NOCOMPRESS turns off compression and overrides the new SHARE_COMPRESS logical name. See the separate paragraph on this below. /DEBUG[=n] Turn on extensive debugging messages. /NODEBUG turns off the messages and overrides the new SHARE_DEBUG logical name. See the separate paragraph on this below. /LOG Turn on informational logging messages. Note that some messages used to appear by default. This is no longer the case. To be consistent with the rest of VMS, messages are off by default. /NOLOG will turn off messages and override the SHARE_LOG logical name. /LONGLINES Allows the packaging of files with records longer than 960 characters (the limit in older versions of TPU). However, this is only allowed if running on a version of VMS which is 5.4 or later. /NOLONGLINES will disable the long line support and override the new SHARE_LONGLINES logical name. See the separate paragraph on this below. /PACKAGE_INDEX[=suffix] Creates a package file, suitable for use by file servers, listing the filenames of each part created. /NOPACKAGE_INDEX will disable the creation of this file, overriding the new SHARE_PACKAGE_INDEX logical name. The optional suffix value specifies the file type given to the created file. See the separate paragraph on this below. /PART_SIZE=nn Specifies the part size to be generated. Overrides the SHARE_PART_SIZE logical name. Qualifiers may appear anywhere in the command line (although, because VMS_SHARE is a DCL procedure, they may not appear directly after the command verb!). 2.2 Support for Various Record Formats This version introduces support for various record formats, including VARIABLE length records with all the carriage control options, FIXED length records of any size up to the maximum supported record length, and each of the STREAM record formats. Previous versions have supported only those files with a variable length record format and CR Carriage control attributes (the standard file type created by TPU when it writes a file). This mechanism is implemented by recording the file attributes in the share file, and passing them to the CONVERT utility for processing once the file has been decoded and written out in its normal variable length format. Using this facility, it is now practical to directly package .OBJ and .EXE files; if the long line support is selected, it is possible to directly package backup savesets, thus allowing any type of file to be packaged. BACKUP requires a minimum /BLOCK_SIZE specification of 2048, which in turn requires the long lines support. Therefore this is possible only with VMS 5.4 or later at both the creation and the unpacking ends. See the paragraph on long lines support below. [Thanks go to Mark Pizzolato - mark@infocomm.com - for this suggestion] 2.3 Optional Run-Length Encoding of Files For files which contain long runs of similar characters, the new run-length encoding facility may be of help in reducing the size of the share files. A simple algorithm is applied to the files so that runs of 6 or more characters are compressed to a 5 character sequence thus: &nnZZ where "&" is a new flag character that introduces a run-length compression sequence, "nn" is the length count as a 2-digit hex number, and "ZZ" is the 2-digit hex encoding of the repeated character's ASCII code. Although the minimum run length is 6 before compression is applied, a special check is made for a run of 2 or more characters that would ordinarily be quoted in the existing form of: `ZZ as this results in a smaller sequence of characters. Because this feature can be time consuming, it is optional. The normal default is NOCOMPRESS. A new logical name SHARE_COMPRESS may be defined to alter the default. The /[NO]COMPRESS qualifier on the command line may be used to override the default. [Thanks go to Mark Pizzolato - mark@infocomm.com - for this suggestion] 2.4 Long Lines Option VMS_SHARE now supports files with record lengths up to 65535 bytes. This is due to the later versions of TPU (since VMS 5.4) supporting much longer records. Older versions of TPU restricted the maximum line lengths to 960 bytes. In order to maintain compatibility, TPU uses the old length of 960 as the maximum record length by default. The new /LONGLINES qualifier requests that the long line support be used, allowing records up to 65535 bytes to be packaged. The use of /LONGLINES has some caveats of course. Firstly, it can only be used if VMS_SHARE is running on a version of VMS 5.4 or greater; if not, an error message is issued if long line support is selected. Secondly, the generated share file will require at least VMS 5.4 to unpack (normally only VMS 4.4 or greater is required); this is because the TPU code used to unpack must also have the long line support, which is not available in earlier versions. To summarize: Without /LONGLINES (The default): The minimum VMS required to run is 5.0 Records are restricted to 960 characters The minimum VMS required to unpack is VMS 4.4 With /LONGLINES: The minimum VMS required to run is VMS 5.4 Records can be up to 65535 characters The minimum VMS required to unpack is VMS 5.4 Long line support is off by default. The default can be modified by defining the new logical name SHARE_LONGLINES to either TRUE or FALSE as appropriate. This default can be overridden by the presence of the /LONGLINES qualifier on the VMS_SHARE command line - /LONGLINES enables support, /NOLONGLINES disables it. 2.5 New Logical SHARE_EXCLUDE_FILES In addition to excluding files from the packing process by name, type and directory, a new logical name SHARE_EXCLUDE_FILES can be defined to exclude specific filenames (I.E. combined NAME and TYPE). For example: $ define SHARE_EXCLUDE_FILES "NAME.TMP,X.DAT" will exclude any file whose name is "NAME.TMP" or "X.DAT" in any of the directories which VMS_SHARE is searching. [Thanks go to Joe Meadows - JOE@FHCRVAX.BITNET - for this suggestion] 2.6 New Format for Share File Part Names To make it easier to keep track of the parts that make up a share file, the naming convention has been changed. Previously it was "sharefile'n'" where "n" was a part number (from 1 upwards). The new format changes the suffix to be "nnn-OF-mmm" where "nnn" is a part number and "mmm" is the total number of parts generated. Typical filename might be: SHARE.078-OF-100 meaning part 78 out of a total of 100 parts. The parts are renamed to this format once the TPU packing code has completed. [Thanks to Dick Munroe - munroe@dmc.com - For this suggestion] 2.7 Automatic Creation of Share Part Index File Many mail based file servers recognize a `package' file. This is a file which contains the names of other files to be sent. The user can specify this single file and receive all the files whose names are contained in the package file. Such a package file can now be created automatically. It is not created by default. The new logical name SHARE_PACKAGE_INDEX can be defined to a TRUE or FALSE value to modify the default (TRUE causes its creation). The new /[NO]PACKAGE_INDEX qualifier overrides any default. The file so created has the form: name.$PACKAGE where `name' is the NAME part of the output sharefile and `$PACKAGE' is the fixed file extension. It is created in the same directory as the sharefile parts. Its contents are a list of the individual parts of the sharefile, one per line. Only the NAME and TYPE of the sharefile parts are listed. The suffix ".$PACKAGE" is the default; this can be overridden by specifying it as the value of the /PACKAGE_INDEX qualifier. For example: /PACKAGE_INDEX=.LIBRARY which would change the default from ".$PACKAGE" to ".LIBRARY". The logical name SHARE_PACKAGE_INDEX can also be defined with a suffix value, rather than a TRUE or FALSE value, and this will modify the default to be that suffix. 2.8 Debug Facility The debug facilities have been revised in a number of ways. First, debugging is now controlled via a qualifier on the command line or, if not specified, by the setting of the new logical name SHARE_DEBUG. Second, several levels of debugging are recognized, from 0 (/NODEBUG) through 1 (/DEBUG) through n (/DEBUG=n) up to a maximum n of 3. Each level gives progressively more debugging information and includes all the debugging output for lower levels. Higher levels of debugging (greater than 1) are intended for really difficult problems. The SHARE_DEBUG logical can be defined with either a TRUE/FALSE value or an integer value. FALSE is equivalent to an integer value of 0 and specifies no debugging (the default), TRUE is equivalent to a debugging level of 1. Values higher than 3 are treated as 3, values lower than 0 are treated as 0. Finally, debugging may only be specified if the user has SYSPRV. If not, the debugging level is set to 0 regardless of what was specified. The debugging output has also been made more compact (I.E. less verbose) than in previous versions. 2.9 New logicals SHARE_UNPACK_LOG and SHARE_UNPACK_VERIFY In previous version, the SHARE_LOG and SHARE_VERIFY logical names affected both the VMS_SHARE utility and the unpacking of the packaged share file. For example, defining SHARE_VERIFY to TRUE would cause both to be verified when run. This release introduces two new logicals which affect ONLY the unpacking of the sharefile - SHARE_UNPACK_VERIFY turns on/off verification of the code and SHARE_UNPACK_LOG turns on/off logging of each file unpacked. The old logicals SHARE_LOG and SHARE_VERIFY still exist but affect only the VMS_SHARE utility itself. Note: Because logging is off by default in both VMS_SHARE and the generated share file, no messages will be produced unless the appropriate logical is defined to be TRUE (or the /LOG qualifier is used on the VMS_SHARE command). 3. BUG FIXES The following bugs have been fixed. 3.1 Removal of Spurious Dot in PROCESSING Message The unpack code gives the message `Processing file xxxx.'. The extra dot at the end of the message was fouling attempts to cut/paste the filename with a double click in a windows environment and has been removed. [Thanks go to Claude Barbe - barbe@sdrvx2.sinet.slb.com - for this amendment] 3.2 Part Splitting The part splitting algorithm has been modified to more accurately compute the size of a part written to disk. The part generating code has been generally recoded to remove the possibility of a final empty part being created when the data exactly fills the buffer. 3.3 Coding Overhaul Most of the VMS_SHARE code has been extensively overhauled and rewritten so as to be more reliable and check for more things. The general approach to the problem is similar to previous versions however. 3.4 Character Encoding and Line Wrapping A particular area of concern in previous versions has been the method of encoding characters such that intermediate lines created do not exceed the maximum record length imposed by TPU. This has been solved by performing all encoding in a single pass over the buffer and taking whatever action is necessary as each character is examined. A side effect of this means that the built-in SEARCH cannot be effectively used and buffer encoding is considerably slower, even without selection of the run-length compression code. However, the benefits of the new features and the increased reliability should compensate for this. Future releases will try to improve on the speed of packing. 3.5 Long DCL Lines Most of the lines of DCL created in the share file are less than the maximum line length of 79 characters. However, it was previously possible for the CALL UNPACK line, generated for each file, to extend past the end of this maximum. This could happen for very long file names and, with the addition of file attribute recording in this release, is more likely to occur. To avoid the problem, the line length is now checked between each parameter and a DCL continuation inserted if necessary. The check is fairly simple minded and a really long filename could still defeat it but most files should pass through unscathed. Note: This change does introduce a potential problem at the part splitting stage. Should the part split fall between a line and its continuation, the inserted part separators will cause a failure at unpacking time. The current version makes no attempt to deal with this problem since it is fairly unlikely, requiring both a long filename and a part split falling on that line. A future release may solve the problem. In the meantime, if it does occur, the recipient of the share file should edit out the part separators by hand. 3.6 Verification During Share File Unpacking Due to a coding error, defining the SHARE_LOG logical to verify the unpacking of the share file did not work correctly. In fact, defining it to TRUE had the opposite effect of turning off the messages and vice versa. The default action was to display these messages and this worked correctly. 4. CHANGES IN OPERATION The following major changes in operation have occurred. 4.1 Minimum VMS Requirements Due to the use of a number of features of DCL that gave recently been introduced (mainly the block IF, used for clarity of code), the minimum version of VMS required to encode files is now version 5.0. Attempts to use the package on earlier versions will result in an error message. However the generated share file will unpack on all versions from VMS 4.4 onwards; if the /LONGLINES qualifier is used, then the share file will unpack only on versions of VMS from 5.4 onwards. It is felt that the vast majority of users are now running at least VMS 5.0 and therefore the downside is minimal. These users will still be able to unpack share files created without the /LONGLINES option. 4.2 Encoding of Space characters In previous versions, the space character was not treated as a troublesome character. In practice, many files may have leading or trailing blanks removed or added on their journey through the network. This version treats blanks as quotable characters to prevent this problem. As a consequence, the code which used to quote spaces if they appeared at ends of lines has been removed and the code to determine where to split lines is less complex; there is no need to hunt for the last non-blank. This amendment reduces the complexity of the unpacking code in the generated sharefile as there is no need for an additional pass over the data to sort out any trailing blanks. Any blank characters encountered while unpacking are simply ignored. Share files may be slightly longer with this mechanism but choosing run-length encoding should reduce the size due to long runs of blanks being found in most text or program files. 4.3 PAKMAIL Updated The PAKMAIL utility, supplied with VMS_SHARE to mail out all the parts of the package has been updated to reflect the new part format of nnn-OF-mmm. 5. DOCUMENTATION CHANGES The user guide and help file have both been updated to reflect the new support for record formats, run-length compression, the new command-line parameters, and the other miscellaneous enhancements. The technical information guide has been amended slightly for similar reasons. 6. TESTING This version of VMS_SHARE has only been tested on VMS 5.5-1 but is believed to work on all versions from VMS 5.0 to VMS 5.5-1. However, it is not supported formally on any version of VMS other than the one on which it has been tested. The share files generated by VMS_SHARE are believed to unpack correctly on all versions of TPU from VMS 4.4 onwards unless the /LONGLINES option is selected, in which case VMS 5.4 is the minimum release needed.