From: MERC::"uunet!CRVAX.SRI.COM!RELAY-INFO-VAX" 8-JAN-1993 21:06:26.23 To: INFO-VAX CC: Subj: Announcing VMS_SHARE release 8.2 Dear All, VMS_SHARE 8.2 is now available. This includes a number of new features and revised documentation. The appropriate sections of the latest release notes are attached below, so you can determine if you want a copy. I'm grateful to those of you who have suggested new things and supplied bug fixes - I believe I've incorporated all those received to date, in one form or another but please let me know if you feel I've not handled your suggestion correctly. Any new suggestions or bug reports are always welcomed, particularly as this is a major release where quite a bit of recoding has taken place. To summarize VMS_SHARE's function for new readers, it is designed to package a series of files into a multi-part sharefile that can be directly mailed across the network. Once received, the multiple parts can be appended to form a single file. When this file is executed as a command procedure, it recreates the original files. The packaging is designed to circumvent the corruptions that mail systems can introduce and should allow all types of sequential file, binary or text, to be mailed successfully. Copies may be obtained from the mail server as follows: Send the one line instruction: SEND VMS_SHARE.PACKAGE in the body of a mail message to: VMSSERV @ UK.AC.KCL.CC.OAk We're on the UK JANET network; if you're on something else, here's a selection of addresses to try: VMSSERV%OAK.CC.KCL.AC.UK @ NSFNET-RELAY.AC.UK From Internet VMSSERV%OAK.CC.KCL.AC.UK @ UKACRL From Bitnet VMSSERV%OAK.CC.KCL.AC.UK @ UKNET-RELAY.AC.UK From UUCP If all goes well, you will receive the package in 19 parts, shortly afterwards. The mail server may take a day or so to get the files back to you, so please wait a while before retrying. If all else files, mail me directly and I'll post a copy manually. Regards, Andy Harper Kings College London V M S _ S H A R E R E L E A S E N O T E S Version 8.2 Jan 1993 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - This software is copyright but may be freely distributed without charge to anyone. All copyright and ownership notices must remain intact. No warranties are offered as to the suitability of this software for any purpose; any errors arising from its use are entirely the responsibility of the user. (C) Andy Harper, Kings College London, England - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1. INTRODUCTION Version 8.2 of VMS_SHARE is a major release which fixes a number of bugs and makes a number of new features available. 2. NEW FEATURES 2.1 File Exclusion The new /EXCLUDE=filelist qualifier may be used to exclude a set of files from the packaging operation. `Filelist' is a comma separated list of full or partial file name specifications. Each field in a filename specification (Field = DIRECTORY, NAME, TYPE or VERSION part) may be omitted or may be a specific directory, name, type or version in the usual format. Fields which are omitted are assumed to match anything. Files which match the specification are excluded from the packaging operation. Some examples: /EXCLUDE=.FOR Exclude all files with a .FOR file type /EXCLUDE=.EXE,.OBJ Exclude all files with a .EXE or a .OBJ file type /EXCLUDE=DUMMY.DAT Exclude any file called DUMMY.DAT regardless of directory or version /EXCLUDE=;1 Exclude all version 1 files /EXCLUDE=[X]DUMMY.DAT Exclude all versions of the file DUMMY.DAT in directory [X] Wildcards are not permitted, although an `*' can be used in the usual place ONLY to specify a complete field (NOT partial fields). For example: /EXCLUDE=*.FOR Is allowed and excludes all files with a type of .FOR /EXCLUDE=X*.FOR Is NOT allowed as the asterisk is used to match only part of the name field and not the whole field. The default is to include all files specified by the user's parameter list. However the new SHARE_EXCLUDE logical name may be defined with a list of exclusions to override this default. Use of the /EXCLUDE qualifier overrides the setting of this logical name for any given run. Note that this qualifier and logical name makes redundant the following logical names: SHARE_EXCLUDE_DIRS To exclude directories SHARE_EXCLUDE_FILES To exclude specific filenames SHARE_EXCLUDE_NAMES To exclude files with specific names SHARE_EXCLUDE_TYPES To exclude specific file types These logical names are still recognized in addition to the SHARE_EXCLUDE logical name, but the effect is overriden if the /EXCLUDE qualifier is used. They will be removed in a future release of the software. 2.2 New Compression Mechanism Previous releases have supported a simple compression mechanism based on run-length encoding, which is a simple but unsophisticated algorithm for reducing the size of the packaged files by encoding sequences of the same character into a shorter sequence. This version introduces an additional technique based on the algorithm popularly referred to as Lempel-Ziv encoding, where substrings are encoded into a form that points back to a previous occurrence of the same substring. This results in a more significant compression ratio at the expense of more CPU time to package the file. The `official' Lempel-Ziv algorithm has been substantially modified to take into account the additional constraints of this application. As such, it is not as effective as `real' lempel-ziv but appears to do a reasonable job in the majority of files with which it has been tested. As before, selection of this compression technique may be of little benefit in some cases and is very much slower than selecting no compression; therefore it is NOT selected by default. To allow user selection of this mechanism, the /COMPRESS qualifier (and corresponding logical name SHARE_COMPRESS) has been modified to accept a numeric parameter. The value specifies the compression mechanism to be used. Currently: /COMPRESS=0 Specifies no additional compression /COMPRESS=1 Specifies run-length encoding /COMPRESS=2 Specifies the modified Lempel-Ziv compression Other values are not permitted. /COMPRESS by itself is the same as /COMPRESS=1; /NOCOMPRESS is the same as /COMPRESS=0 NOTE: it is NOT intended that this form of compression should supplant that which can be obtained by external means, such as the ZIP utility, which can always be used if required. However, it has the advantage of being easy to use and not requiring the recipient to have any external decompression tools. 2.3 Support for VFC files This release introduces support for the record format VFC, and associated fixed control area. 2.4 Enhanced /LOG qualifier The /LOG qualifier may, like /COMPRESS, now take a numeric parameter to specify the level of logging required - /LOG=n. Permitted values are 0, 1 and 2. /LOG=0 specifies no logging output is to be produced. It is equivalent to /NOLOG. /LOG=1 specifies that minimal logging should be produced, just a summary of each file as it is processed, a final summary of the number of parts generated, and a note about the package index file if one has been requested. It is equivalent to /LOG with no parameter. /LOG=2 specifies that additional logging is to be produced over and above that of level 1. Currently, this produces a summary of the file encoding statistics before and after encoding as an aid in determining the effectiveness of the compression. Previously, this output was part of the /DEBUG facility and therefore required privileges. The corresponding debug messages have been removed and the facility is available to all users through /LOG. Additional log messages and levels of logging may be added in future releases. 2.5 Strip Directory and Version Numbers Recent previous releases have preserved the subdirectory structure of the packaged files and the version numbers. Some older releases did not. Some users have requested an option that allows the older behaviour to be reinstated. As a result, two new qualifiers have been added that allow the user to select whether directories and/or version numbers are retained in the sharefile. The new qualifier /[NO]DIRECTORY selects the [no] directory structure preservation option. If preserved, then the directory structure will be recreated when the share file is unpacked. If not preserved, then all files will be created in the directory current at the time of unpacking. The logical name SHARE_DIRECTORY may be defined to specify the default, with /DIRECTORY being the default otherwise. The new qualifier /[NO]VERSION selects the [no] version number preservation option. If preserved, then the version numbers are retained when the share file is unpacked and a file with the same version number will cause the unpacking of that particular file to be skipped. If not preserved, then a new version of the file will be created regardless, with a warning message if a lower numbered version already exists. The logical name SHARE_VERSION may be defined to specify the default, with /VERSION being the default otherwise. [Thanks to Wolfgang Moeller - Moeller@gwdgv1.dnet.gwdg.de - for this suggestion] 2.6 DEBUG Now Shows Rejected Files If a file which matches the user specified input files is not selected because of the value of the /EXCLUDE, /AFTER or /BEFORE qualifiers, selecting the /DEBUG qualifier will show which of these qualifiers caused it to be rejected. Level 2 debug is required for this. 2.7 New PAKMAIL Utility The PAKMAIL utility has been extensively revised. In particular, it now recognizes an optional list of parts as a paramter. This allows a user to send a subset of the full package should, for example, a number of users fail to receive some of the parts. The User guide and on-line help give further details of this. [Thanks to Richard Levitte - Levitte@ttt.kth.se - for this suggestion] 3. BUG FIXES 3.1 Debug Messages In some early releases of version 8.1, some of the debug messages were always active. This was due to a faulty definition of their debug level (as 0 rather than 1) internally. The debug level associated with these messages has been corrected. 3.2 PAKMAIL The PAKMAIL utility, used to mail out parts of a package, was modified in version 8.1 to reflect the new part format of nnn-OF-mmm. A bug in the code prevented this working correctly when the number of parts exceeded 9. This has now been corrected and works for any number of parts. 3.3 Check for Same Subdirectory The check on whether a file was in the same subdirectory was faulty, leading to a DCL error message about undefined symbols rather than the correct message. This has been corrected. [Thanks to Wolfgang Moeller - Moeller@gwdgr1.dnet.gwdg.de - for this bugfix] 3.4 File scanning In some early version of VMS (around 5.3 and earlier), there was a dcl bug that prevented nested loops containing gosubs from working correctly. After several iterations, DCL would fail to find labels that were definitely present. This situation could arise in that part of the code which scans the user's file specifications and locates matching files. To avoid the problem, the code has been split into two sections, circumventing the bug. Note that the bug appears to no longer exist on VMS 5.5. [Thanks to Foetes Macrides - MACRIDES@WFEB2.BITNET - for this bugfix] 3.5 Check for same device The check on the filename to make sure it exists on the current device was inside rather than outside a loop. This resulted in one message for each file that matched a wildcarded file spec on the device. The check has been moved outside the loop and is now issued once when the wildcarded pattern is first used; the pattern is not used if the device is invalid. Note that this check is suppressed if the /NODIRECTORY option is requested. 3.6 Check for invalid file specifications The input file specification supplied by the user is now checked for valid syntax. Any invalid file specifications cause an abort. 3.7 Check for missing parameters in non-interactive mode If any of the required parameters are not supplied on the command line, the user is prompted for them. However, if the mode of the process is not INTERACTIVE, the user cannot be prompted; therefore an error message is issued in such modes. 3.8 Problems with checksumming Due to the problems described in section 6.1 of these release notes, the checksum mechanism cannot always be used. Under such circumstances the file cannot be packaged either. Previous releases inserted a check into the created share file on the assumption that the checksum might fail but the packaging would not. However, this cannot happen so the appropriate code segment has been removed from the created share file. Should a file not be able to be checksummed at the packaging stage, an appropriate message is issued and the file is ignored. 3.9 Check on Output Directory/FileName If the output directory was non-existent, or there was a syntax error in the output file specification, the share file parts were created in the current directory with strange names. A check has now been included to validate the output file. 3.10 Checks on Qualifier Formats More checks have been added to the qualifier parsing code to ensure that qualifiers that need values have them, and that those which don't require them do not have them specified. 3.11 Input File Patterns A warning message is now issued if any of the input file specifications do not match at least one file. The message is given for each pattern that fails to match. 3.12 Parameter File Creation Additional checks are now made on the creation of the parameter file, to ensure that all parameters have been correctly written. 3.13 Null file types Null file types were processed badly in some places; all known instances of this have been corrected and should now work properly as with any other file type. [Thanks to Richard Levitte - Levitte@ttt.kth.se - for this bugfix] 4. CHANGES IN OPERATION The compression routines have all been completely re-written. The only new code is that dealing with the LZ encoding. 5. DOCUMENTATION CHANGES The user guide has been completely re-written. Details on the new features have been added and descriptions of all the qualifiers have been moved to a separate qualifier reference section. The HELP file has been updated with details of the new qualifiers (/EXCLUDE, /DIRECTORY and /VERSION) and the new logical names corresponding to them. Most of the logical name descriptions have been rewritten so that the reader is referred to the corresponding qualifier rather than duplicating information. 6. MISCELLANEOUS NOTES 6.1 Record Length Problems The release notes for version 8.1 outlined some problems with long records, whereby TPU and the CHECKSUM utility were failing to read certain files with longish lines in them. The reason for this is now clear, although a solution has not yet presented itself. Essentially, each file header contains TWO fields describing the size of each record; the first is the maximum record size, which contains the longest line that may appear in the file - a zero value here signifies no upper limit on the length - and is used by many system utilities, TPU included, to determine the size of the record buffer; the second is the length of the longest record that is currently written to the file. The problem arise when each of these conditions is true: (a) The MRS value is zero, which causes TPU and CHECKSUM to allocate default record buffer sizes of about 2048 bytes. (b) The longest record in the file exceeds the default record buffer length. Unfortunately, DCL's lexical function f$file_attributes only allows a procedure to see the MRS value and NOT the longest record value. It is therefore impossible to predict in advance which files are going to cause problems during packaging. The CHECKSUM utility suffers from a similar problem and consequently will not checksum files with an MRS value of 0 and a longest record greater than 2048 bytes. It is hoped that future developments in VMS may allow this problem to be circumvented. In the short term, the simplest workaround is to use the CONVERT utility to create a new version of the file with the MRS value set to the size of the longest record. The longest record can be determined by using DUMP/HEADER/BLOCK=C:0 file and looking for the RMS attributes section. Once the size of the longest record is obtained, call it `nnn', use the CONVERT utility as follows: CONVERT/FDL=SYS$INPUT file file RECORD SIZE nnn ^Z This new copy of the file will then be correctly processed by VMS_SHARE 7. TESTING This version of VMS_SHARE has only been tested on VMS 5.5-1 but is believed to work on all versions from VMS 5.0 to VMS 5.5-1. However, it is not supported formally on any version of VMS other than the one on which it has been tested. The share files generated by VMS_SHARE are believed to unpack correctly on all versions of TPU from VMS 4.4 onwards unless the /LONGLINES option is selected, in which case VMS 5.4 is the minimum release needed.