The VAX Notes Notefile A Technical Description Dale E. Coy Preface VAX Notes is seeing increasing usage within the DEC community. With this increasing use, there is a need for additional tools, and potentially for user-written applications to interface with the Notefile. Although a callable interface may eventually be available, it may not meet all users' needs. In any case, an understanding of the Notefile format and utilization is useful. Using the standard tools shipped with VMS, and experimenting with VAX Notes, it is possible to develop an understanding of how the Notefile is organized and used. The research leading to this description was partially supported by the DECUServe system. Valuable contributions were made by a number of DECUServe members. I want to particularly recognize the assistance of Jon Jones and Bob Denny. ** CAUTION ** This description is based on exploration of VAX Notes V1.2 and V2.0. None of it has been confirmed by DEC, and it's obviously not supported by DEC. I believe that it is accurate, and essentially complete. However, if you have additional information, or if you find out that something is wrong, I would appreciate hearing about it. A Note About This Description In the material below, the position of bytes in a record is described by the index (as opposed to the offset). That is, the first byte in the record is Byte 1. Where not explicitly indicated, numbers are given in decimal. The tools I normally use are VAX Notes itself (for experimentation) and the EVE editor (actually WPE). My "picture" of the Notefile is colored by what I see on my screen. This is a useful abstraction, although it's only one view. Things may "look" different using DUMP or some other utility. File Organization The user's personal notebook file (NOTES$NOTEBOOK.NOTE) and the Notefile both have the same organization and record structure. However, we will concentrate on the Notefile here. The Notefile is usually named XXXX.NOTE, where XXXX is the Conference Name. However, the filename is arbitrary and logically has nothing to do with the Conference name. The VAX Notes Notefile is an Indexed File, and has 3 Keys. The records are variable-length, maximum 1024 bytes. The first 76 bytes of each record is considered to be the Key Area. Key 0 - a Binary key, 4 bytes, located at Bytes 1-4. Unique within the Notefile (no duplicates), changes are NOT allowed. The content is a number related to the UID. Key 1 - a Binary key, 4 bytes, located at Bytes 73-76. Not unique (duplicates allowed). If non-zero, the content is a representation of the Note Number (2 bytes for Topic Number, 2 bytes for Reply Number). In a Notefile from VAX Notes V1.x, this key is marked "changes not allowed". In a V2.x Notefile, changes are allowed (implementation of new feature in VAX Notes V2.0). Key 2 - a String key, 64 bytes, located at Bytes 5-68. Mostly "wasted space" (nulls) in the Notefile. Not unique (duplicates allowed), and changes are allowed. Only used for Member, Keyword, and Node records (and even then, mostly nulls). Obviously, that leaves Bytes 69-72 for a "spare key" someday. These bytes always seem to be nulls, except for a usage of Byte 69 as a "continuation indicator". The remainder of the record, starting with Byte 77, is DATA. Note: for the "new hacker" looking at the Notefile for the first time with EVE... What you see on your screen for a "binary key" is in inverse order from the value of the key. For instance, if the 4-byte binary key is (Hex) 40 00 00 01 then the four bytes on your screen, when "read" in Hex, will appear in the order: 01 00 00 40 The Major Areas of a Notefile +--------------------------------------------------------------------+ | Record 1 - Conference Data and Control Part of Zero Area| | Record 2 - Conference Title | +--------------------------------------------------------------------+ | The rest of the Zero Area (Key 0 = 00 00 xx xx) | | Contains Records describing Moderators, Keywords, and Nodes | | | +--------------------------------------------------------------------+ +--------------------------------------------------------------------+ | The 40-Area (Key 0 = 40 xx xx xx) | | Contains One Record for each Note, with data for the | | Note Header. | | | +--------------------------------------------------------------------+ +--------------------------------------------------------------------+ | The 80-Area (Key 0 = 80 xx xx xx) | | Contains one or more records for each Note, with data for the | | text of the note. | | | +--------------------------------------------------------------------+ For most conferences, the Zero-area will be fairly small. (Exception: when there are a lot of Members and/or Keywords). The 40-area is larger: exactly one record per note in the conference. The 80-area is usually even larger. The amount of text per record is 948 bytes or less. In other words, the average "screen" of information in VAX Notes takes about one of these records. A General Discussion of "Unique IDs" The Notefile is organized on the principle of a Unique ID (UID) for each entry. Understanding of the principle is needed for the rest of the discussion. *** CAUTION *** The following discussion is NOT exactly how UIDs work in VAX Notes. The next section will describe that. The purpose here is to give a GENERAL idea of the principle. Notes is inherently a non-relational database. Because of the application, it MUST have a UNIQUE key for each entity (Note). The UID (Unique ID) is that key. The UID is assigned at note creation, it is never changed, and there must never be a chance of duplicates in a conference. Because extreme measures are taken to assure it's uniqueness, a LOT of Notes code can ) and does assume that it's unique. It is sufficient to keep a value somewhere in the conference "header" that is the "Last UID Written". When you press Return at the question "Add this reply" (or whatever), an "atomic" operation is done to assign the next UID, and increment "Last UID Written". (obviously, being very careful to lock things properly - which was why we had to have a new version of Notes for SMP machines). Digression: Notice that the Last UID Written (LUW) always increases, even though deleting Notes may cause the quantity of visible notes to decrease. Notice that a copy of "LUW" can be kept in your individual notebook. When you do an update, if the Conference's "LUW" is the same as the one your notebook remembers, there are Zero unread notes. Your notebook's remembered "LUW" can Never be greater than the one in the conference. ('never say never'). If the Conference "LUW" is greater than your notebook remembers, then Notes THINKS (at the Conferences display) that there are N unread notes (simply, the difference between the two). So - if you think about it - this explains seeing "N unread notes", but opening the conference and nothing is there. So - what is the relationship of "note number" (as in 2.0, 2.1, etc.) to the UID? Absolutely none. The "note number" is another key in the record. In other words, re-numbering a note has NO effect on the UID. In VMS Terms: The UID is equivalent to the File ID. You can rename this thing all you want - the File ID won't change. Where is this leading? o The UID for *EACH* note *MUST* be *UNIQUE* within a conference. o VAX Notes will not allow you to change (or even see) the UID. How do UIDs Really Work in VAX Notes? There are two different groups of UIDs. This does not violate the principle that they must be unique: One group starts at 00 00 00 00, and the other starts at 40 00 00 00. In practical terms, the groups will never overlap. The UIDs are used as KEY 0 (Bytes 1-4) in the Notefile. UID0 The first group, which we're calling UID0, is used in the Zero Area. It provides the KEY 0 value for records describing Members, Keywords, and Nodes. The first two records in the conference are special - and have UID values of 00 00 00 00 and 00 00 00 01. The UID0 values then begin with 00 00 01 01, and count upward. The "Last Used" value for UID0 is stored in Record 1, Bytes 89-92. As discussed in the general description, UID0 values are never re-used. The Note UID The Note UID is used to identify entries in the Note Header ("40") section and, when transformed, in the Note Content ("80") section. As we will see, one UID is sufficient for both purposes. The "Last Used" value for this UID is stored in Record 1, Bytes 93-96. The Note UID is referred to in the rest of this discussion as just the UID - or sometimes as the 40-style UID. UID values begin with 40 00 00 01, and count upward. These UIDs are used directly as keys for the NOTE HEADER records. Because it would be unacceptable to limit the note text to 900 bytes or so (if we only had one text record), we need some way to have more than one text record per header record. Using a transform of the UID, we can create other unique IDs that fit this requirement. Transforming the UID for use in the 80 Area The records in the "80 area" contain the NOTE CONTENT (text) information. There is ONE OR MORE of these records PER NOTE. The record is usually present even if there is no text. KEY 0 binary values in the 80 area are a TRANSFORMATION of the UID. The transformation is done to provide another set of unique keys, and to allow up to 128 text-records per Note. (Overflow is handled by continuing with the next UID). The transformation may be accomplished in the following way: Assume that the Note UID is 40 00 00 01 (This is the UID for the first note posted; obviously Note 1.0 - but the method is general). UID 40 00 00 01 Remove the High Byte 00 00 01 Shift Left 7 (or multiply by 128) 00 00 00 80 And turn on the High Bit 80 00 00 80 Additional Text for this note in 80 00 00 81 records with succeeding keys: 80 00 00 82 ... Let's try another example, using UID 40 00 00 02 UID 40 00 00 02 Remove the High Byte 00 00 02 Shift Left 7 (or multiply by 128) 00 00 01 00 And turn on the High Bit 80 00 01 00 Additional Text for this note in: 80 00 01 01 Very clever............ What happens if we have text SO LONG that we need more than 128 records? The Key just continues to be incremented: 80 00 00 FE 80 00 00 FF 80 00 01 00 80 00 01 01 ... Of course, VAX Notes is smart enough to realize that doing this implicitly "uses up" UID 40 00 00 02, and properly handles the situation so that we won't have duplicate keys. In effect, the UIDs are just skipped - a note header record with that key is never posted. Key 1 - The Note Number KEY 1 (Bytes 73-76) is the Note Number of the note. This Key is used only in the Note Header (40) and Note Content (80) records. (It's Null in the Zero Section). This key allows you to find all records which are associated with a particular note. The Note Number can be interpreted as a 4-byte binary value, or as two 2-byte values. The full (longword) key is useful lets us find all parts of a single note. Bytes 73-74 are the "reply number" word (may be zero, of course). Bytes 75-76 are the "topic number" word. For example, if the note number is 5.1: 4-byte value: 00 05 00 01 Two 2-byte values: 00 05 and 00 01 Of course, these are really Hexadecimal numbers. Therefore, the maximum Topic number and the maximum Reply number are both 65535 (Hex FFFF). You will notice that there is NO mention of UID here. To re-number a note, it is only necessary to search for and change all of the matching KEY 1 values. Obviously, it would be good practice to check that re-numbering will not result in TWO sets of notes with the same KEY 1 values. VAX Notes V2.0 provides a moderator command to re-number notes. SET NOTE notenumber/NOTE_ID=new-noteid KEY 2 - Some Text KEY 2 (Bytes 5-68) is used for rapid lookup of Keywords, Members, and Nodes (Nodes are an addition in V2.0). This Key is used only in the Zero Area. In other sections, it is Null. KEY 2 is constructed as follows: The first byte of the key (Byte 5) is: Hex 04 if this is a Keyword Record. Hex 07 if this is a Member Record. Hex 08 if this is a Node Record. Hex 00 if this is a Continuation Record. This is followed by a "translated" form of the Keyword , the member's Membername, or the Nodename. The translated form is exactly the same length as the "plain text" form. If you need the length of the item, it will be found in Byte 79 (not in the key) - because of the way these records are constructed. Translation is done using the following table: Input: ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz Translated: ABCDEFGHIJKLMNQRSTUWXYZ[\] ABCDEFGHIJKLMNQRSTUWXYZ[\] ^^^^^^^^^^^^ ^^^^^^^^^^^^ Input: [\]^_`{|}~ @$%&*<=;.?'/,!#()+>:-" Translated: abcdefghij @$%&*<=;.?'/,!#()+>:-" This group is unchanged Space translates to Space. Note: Earlier versions of VAX Notes (somewhere before V1.2) used the plain text in Key 2, rather than this translation. A field in Record 1 of the Notefile lets you know if this is the case. The remainder of Key 2 (through Byte 68) is filled with Nulls. Notefile Record 1 - Conference Data & Control Record 1 contains control, dynamic, and static information about the entire Conference. Record 1 is the ONLY record in the Notefile that: o Has a guaranteed length (124 Bytes) o Content is invariably in the same place. All Keys for Record 1 are Hex 00 (Null). That is, it's UID is 00 00 00 00. All bytes 1-76 are null. Byte(s) Contents 77-80 00 00 2C 00 (if interpreted as a binary: 00 2C 00 00) This appears to be a constant (Notes 1.2 and 2.0) Hex 2C is the character "," 81-84 03 00 00 00 (if interpreted as a binary: 00 00 00 03) This is probably a number indicating the notefile format. Appears to be a constant (3) for VAX Notes V1.2 and V2.0. A value of 2 has been observed for an older notefile format. These values would allow VAX Notes to process differently, based on the "age" of the file (that is, V1.2 can recognize and process V1.0 Notefile format). As an example of the type of difference that may be significant, the notefile with a value 2 in this field has KEY 2 in "plain text" form, rather than the "translated" form described elsewhere. 85 Flags: 1 = Members-Only Conference 4 = ENABLE Keywords (/norestrict=keywords) 8 = NoWrite 16 = Reply_Only (Hex 10) (The usual rules apply: Hex 1D turns 'em all on) VAX Notes V2.0 supports a new conference restriction, set by the Moderator command: MODIFY CONFERENCE/REPLY_ONLY 86-88 Null (so 85-88 may be interpreted as a binary: 00 00 00 nn) 89-92 Binary value: Last Entry in the Zero Section. Could be called "UID0". Typically interpreted something like 00 00 01 05 (if the last entry was #5). 93-96 Binary value: The "real" UID. Last Entry in the 40-section. Typically interpreted something like 40 00 00 09 (if there have been 9 notes posted). 97-100 Binary value: Total Notes Count. This is the "present value". If notes are deleted, it decreases. Might be 00 00 00 09 (or anything less) in our example. Notes displays this as "Total Notes". 101-104 Binary value: Highest Topic Number. "present value". Notes displays this as "Total Topics". It may exceed the actual total topics, if intervening topics have been deleted. 105-112 8 Bytes of Date/Time. VMS Date Format. This is the "Date the Conference was Created" and never changes. 113-120 8 Bytes of Date/Time. VMS Date Format. This is the "Date/Time the Conference was Modified". Any action which changes the conference content (including things like deleting notes) updates this. 121-124 Binary value: UID Number (40-section format) of the Last, Highest Note Deleted. When the Conference is created, this field contains 00 00 00 00. If the last note in this example were deleted, the field would be 40 00 00 09. It appears that this will not be subsequently changed if a lower UID note is deleted, but will be changed when a higher UID note is deleted. All values in Record 1 are "fixed" at conference creation, except for the following fields/conditions: Byte(s) Modified when: 85 change a flag 89-92 ADD a moderator or keyword 93-96 ADD a note (write or reply) 97-100 add/delete any note 101-104 add/delete the HIGHEST-NUMBERED Topic note 113-120 change the conference in any way 121-124 delete a note (sometimes) Type-Length-Value (TLV) Coding VAX Notes uses TLV coding for the data areas of all records except Record Number 1. Before discussing the rest of the records, a discussion of TLV Coding is needed. TLV Coding is an approach which allows putting necessary information in a record, without using the fixed-field method used in Record 1. TLV Coding is particularly useful when NOT ALL of the fields occur for every record. The TYPE of information is encoded in the first 1 or two bytes, followed by the LENGTH, and then the VALUE (information). The VAX Notes distribution includes a file giving all of the TLV "Tag" values (as well as other constants). The general approach to TLV encoding is as follows: TYPE: Encoded with 1 or 2 byte codes. If the Tag Value is less then or equal to Hex 1E, the TYPE byte consists of the Tag Value Plus Hex C0. (Example: Tag NOTES$K_NOTE_AUTHOR = Hex 06. It's TYPE coding is Hex C6 If the Tag Value is Hex 1F or greater, the TYPE is encoded as TWO bytes: Hex DF followed by the Tag Value. (Example: Tag NOTES$K_NOTEFILE_TITLE = Hex 3C. TYPE coding is Hex DF3C LENGTH: The length of the following data is encoded with 1-to-3-byte codes. Length Format Coding -------------------------------------------------- NULL/0 1-byte format Hex 0 or Hex 80 (VAX Notes uses 0) 1-127 1-byte format Length (8 bits) 128-255 2-byte format Hex 81 followed by length (8 bits) 256-65535 3-byte format Hex 82 followed by length (16-bits) VALUE: Follows immediately, and must be precisely the length indicated. For Records following Record 1, VAX Notes strings together the TLV tagged fields, starting at Byte 77 in the record. Record 2 - Conference Title Record 2 contains the conference title, creator, and (optionally) the conference notice. KEY 0 is the binary value 00 00 00 10 One record is sufficient to contain the data, so this is the only key value. KEY 1 is 00 00 00 00 KEY 2 is Null*64 The two or three "TLV Tagged Fields" start at Byte 77. They usually (always?) appear in the following order: Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_NOTEFILE_TITLE 60 3C < DF3C Variable NOTES$K_NOTEFILE_MODERATOR 54 36 6 DF36 Variable NOTES$K_NOTEFILE_NOTICE 55 37 7 DF37 Variable NOTES$K_NOTEFILE_TITLE may be changed using SET CONFERENCE/TITLE= NOTES$K_NOTEFILE_NOTICE may be changed using SET CONFERENCE/NOTICE=. This is an optional field. NOTES$K_NOTEFILE_MODERATOR is, initially, the Node::Username of the conference creator. This field is changed each time a moderator issues a SET MODERATOR command (that is, it's the Node::Username of the last active Moderator). If this moderator deletes his or her Member record (DELETE MEMBER), the information remains in Record 2, and he/she may still act as a Moderator (until another Moderator does SET MOD). In this situation, VAX Notes produces a "you are still a moderator" message. This technique assures that a Notefile cannot be left without any valid Moderator. Member, Keyword, and Node Records The records in the "zero area" after Record 2 contain the Conference Members, Keywords, and Nodes. These records appear in chronological order of addition, so the Member, Keyword, and Node records are intermixed. Node records are a feature added in VAX Notes V2.0 KEY 0 binary values start at 00 00 01 01 This is the "UID0". If a record with this specific key is present, it will be Record 3, and will be the Moderator record automatically added for the conference creator when the conference was created. KEY 1 is 00 00 00 00 (Bytes 73-76) KEY 2 is constructed as follows: The first byte of the key (Byte 5) is: Hex 04 if this is a Keyword Record. Hex 07 if this is a Member Record. Hex 08 if this is a Node Record. This is followed by a "translated" form of the Keyword , the member's Membername, or the Nodename. The translated form is exactly the same length as the "plain text" form. If you need the length of the item, it will be found in Byte 79 (not in the key) - because of the way these records are constructed. Member Records The Member (or Moderator) record content starts at Byte 77: The first Tagged Field is the "plain text" for the Name (UserName) of the member. The four remaining "Tagged Fields" can appear in any order. Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_USER_NAME 75 4B K DF4B Variable NOTES$K_USER_NODENAME 78 4E N DF4E Variable NOTES$K_USER_MODERATE 74 4A J DF4A 4 NOTES$K_USER_CREATE_KEYWORD 72 48 H DF48 4 NOTES$K_USER_MAIL_ADDR 73 49 I DF49 Variable VAX Notes V2.x NOTES$K_USER_ACCESS_LIST 117 75 u DF75 Variable NOTES$K_USER_WRITE_BYPASS 118 76 v DF76 4 Note: In VAX Notes V1.x, if you do ADD MEMBER NODE::USERNAME, Username appears in NOTES$K_USER_NAME and the NODE appears in NOTES$K_USER_NODENAME. The string NOTES$K_USER_NODENAME may contain several nodes. If there is more than one, the string will be (e.g.,) NODE1,NODE2,NODE3,NODE4... In VAX Notes V2.x, ADD MEMBER places the entire NODE::USERNAME string in the field tagged NOTES$K_USER_ACCESS_LIST, but places additional nodes in the new Node records. NOTES$K_USER_MODERATE, NOTES$K_USER_CREATE_KEYWORD, and NOTES$K_USER_WRITE_BYPASS may be viewed as a character string, or as a longword. If viewed as a character string, characters 01 00 00 00 represent TRUE. If interpreted as a longword, 0 represents FALSE and 1 represents TRUE. MODERATE implies authority to create keywords. NOTES$K_USER_WRITE_BYPASS is valid starting with V2.0, and is associated with the WRITE_REGARDLESS privilege. NOTES$K_USER_MAIL_ADDR may not have been entered. If that is the case, the field will (usually) be present anyway, and will have the values (Hex) DF 49 00. NOTE: The conference creator, automatically added as Record 3, will only have the "N" and "J" fields unless information is later added. Other members always seem to have all fields (N,J,H, and I). Keyword Records The Keyword record content starts at Byte 77: The first TLV Tagged Field is the "plain text" for the Keyword. The remaining TLV Tagged Field is a concatenated list of the UIDs and Note Numbers of notes with this keyword "attached" to them. Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_KEYWORD_NAME 43 2B + DF2B Variable NOTES$K_KEYWORD_X_NOTE 84 54 T DF54 Variable The Length of NOTES$K_KEYWORD_X_NOTE is always a multiple of 9. If you divide by 9, you get the number of notes with this keyword. Each 9-byte value is constructed as follows: Relative Byte 1 Always 00. Relative Bytes 2-5 UID (40-style) of the note referenced. Relative Bytes 6-9 Note Number of the note referenced. The UID and Note Number are interpreted as previously described. Using this information, all notes referring to this keyword may be found. The null value in Relative Byte 1 assures that the reference can always be uniquely found with a search operation. Keyword Continuation Records But, you say, there's not enough room in a 1024-byte record for that many bytes. What do we do about that? If one or more "overflow" records are required, because of the length of the reference string, the following format is used: First, to indicate that a continuation (overflow) record is to be found, Byte 69 is set to Hex 01. This byte is in the "spare" data area - which was previously stated to be all nulls for Keyword records. Secondly, Bytes 77 thru 80 contain the UID0 of the first "continuation" record. The NOTES$K_KEYWORD_NAME tag begins at Byte 81, instead of 77. CONTINUATION RECORDS These are also records in the Zero area. They have a KEY0 value in the usual range. KEY1 is null. NOTICE: Byte 5 (first byte of KEY2) is null. This is the way to identify continuation records: all other records in the Zero area (except records 1 and 2) have a non-null value in Byte 5. NOTICE that MORE than one continuation record may be required, depending on the number of keyword references. If this is NOT the last continuation record, Byte 69 is set to Hex 01. If this is the LAST continuation record for the keyword, Byte 69 is set to null. The 5 bytes 77-81 are not regular data: If additional continuation records are needed for this keyword reference string, Bytes 77-80 contain the UID0 for the "next" continuation record. If this is the LAST continuation record for the keyword, bytes 77-81 appear to have no significance (but they are not data). However, VAX Notes usually has a purpose for things. These bytes probably have some significance that I have not discovered. One additional "rule" about the data area: If Byte 69 is Hex 01, the Nine bytes 77-85 are not part of the keyword reference data. If Byte 69 is Hex 00, the Five bytes 77-81 are not part of the keyword reference data. This structure is a forward-linked list. Byte 69 being Hex 00 is the "end of list" indicator. (Byte 69 being Hex 01 indicates "not end of list yet"). Node Records VAX Notes V2.0 adds an additional record type in the Zero area: the NODE record. This structure makes it easier to handle Members who have different Usernames on different nodes. These members may now have a single Member entry, with access from additional node::user combinations. The Node record content starts at Byte 77: The first TLV Tagged Field is the "plain text" for the Node. The remaining TLV Tagged Field contains a list of USERNAME/MEMBER-POINTER combinations. Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_NODE_NAME 111 6F o DF6F Variable NOTES$K_NODE_X_USERNAME 112 70 p DF70 Variable For NOTES$K_NODE_X_USERNAME, the LENGTH represents the TOTAL length of the following set of USERNAME/MEMBER-POINTER combinations. The USERNAME/MEMBER-POINTER combinations are formed in the following way: m Length of the Username ONLY m+1... USERNAME to be accepted from the node in this record. followed by: 4-bytes The UID0 value that points to the MEMBER record to be associated. Notice that the presence and length of the 4-byte UID0 value are implicit. Only the Total Length accounts for them. Example: If a MEMBER named COY also is granted /ACCESS=NODEP::US166993, the following will happen: A Member record for COY will be constructed (probably with default access EISNER::COY). A Node record will be constructed (or modified) for the node named NODEP. The TLV tagged field NOTES$K_NODE_X_USERNAME will have a string: nUS166993xxxx where n is the number of bytes - Hex 08 and xxxx is the UID0 value of the Member record for COY. A NOTES$K_NODE_X_USERNAME field can contain references to several different Usernames and Member-UID0s. It is possible for one Member to have access from multiple Usernames on the same Node. Note Header Records The "40-Area" contains only the Note Header Records. There is one of these records per Note in the Conference. These records appear in chronological (and UID) order, although deleted notes may cause "holes". I refer to them as the "40 records" because the primary key (Key 0) value for these records is Hex 40 xx xx xx. Each record contains "everything you ever wanted to ask about a note", except for the note text. The records contain the information that appears between the lines at the top of the Notes display screen, and also the "title" information that appears between -< and >- KEY 0 binary values ARE the UID, and start at 40 00 00 01 KEY 1 (Bytes 73-76) is the Note Number of the note. KEY 2 is all Nulls The Note Header record content starts at Byte 77. The first Tagged Field is the "plain text" for the Node and Name (UserName) of the writer of the note. The remaining "Tagged Fields" can appear in any order; the following order is typical. Most of the fields are usually there - but don't count on it. Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_NOTE_AUTHOR 6 06 C6 Variable NOTES$K_NOTE_CREATE_TIME 12 0C CC 8 NOTES$K_NOTE_TITLE 23 17 D7 Variable NOTES$K_NOTE_PEN_NAME 19 13 D3 Variable NOTES$K_NOTE_NUMRECORDS 17 11 D1 4 NOTES$K_NOTE_NUMRESPONSES 18 12 D2 4 NOTES$K_NOTE_WRITELOCK 27 1B DB 4 NOTES$K_NOTE_HIDDEN 13 0D CD 4 NOTES$K_NOTE_NOTEFILE_FILE_NAME 49 31 1 DF31 Variable NOTES$K_NOTE_X_KEYWORD 83 53 S DF53 Variable The NOTES$K_NOTE_AUTHOR string is the NODE::USERNAME. NOTES$K_NOTE_CREATE_TIME is the 8-byte Date/Time the note was written (VMS Date format). NOTES$K_NOTE_TITLE is either the Topic Title or Reply Title. NOTES$K_NOTE_PEN_NAME is the personal name from the writer's profile. NOTES$K_NOTE_NUMRECORDS is the binary NUMBER OF LINES of text. NOTES$K_NOTE_NUMRESPONSES is a binary number shown as the NUMBER OF REPLIES. It appears only for Topic Notes (*.0) The number of replies (which is displayed when you look at a .0 note) is actually the "Highest Reply Number". If replies .1, .2, and .3 have been deleted, but .4 still exists, then the "Number of Replies" will be 4. If this TLV tagged field is absent, the number of replies is assumed to be Zero. NOTES$K_NOTE_WRITELOCK is Hex FF FF FF FF if the Topic is /NOWRITE. Otherwise, the tag may be absent or may be Hex 00 00 00 00. This tag is valid only for Topic Notes. NOTES$K_NOTE_HIDDEN is a binary value (Hex) 00 00 00 01 if the Note is HIDDEN. Otherwise, the tag may be absent or may be 00 00 00 00. NOTES$K_NOTE_NOTEFILE_FILE_NAME is the Text String for the Conference Reference. This string is set using the /CONFERENCE= command. If this string is present, the Note Number displayed is flagged with a "*". IF the note has Keyword references, the data will contain the plain-text of each Keyword. Keywords are concatenated within a tagged field, constructed as follows: The LENGTH given with NOTES$K_NOTE_X_KEYWORD is the TOTAL length of the following string. The string then is composed of LENGTH/VALUE items which are the keywords: n+3 Length of Following Keyword n+4... Keyword Text n+ Length of Following Keyword n+... Keyword Text .... This continues until all of the Keywords have been included. The total length in the TLV includes the bytes for keyword length, as well as the length of all keywords. Continuation Records, as described above, may be an applicable structure here. That would mean a lot of keywords for an individual note, and an example has not been found "in captivity". Note Text Records The records in the "80 area" contain the NOTE CONTENT (text) information. These records appear in chronological (and UID) order. There is ONE OR MORE of these records PER NOTE. The record is present even if there is no text, in VAX Notes Version 1.2 and above. Examination of an old Notefile indicates that this was not always the case. If there is no text (i.e., Lines=0), there may not be an associated note content record. However, V1.2 of VAX Notes appears to always put in this record, even if Lines=0. I refer to them as the "80 records" because the primary key (Key 0) value for these records is Hex 80 xx xx xx. KEY 0 binary values are a TRANSFORMATION of the UID. The transformation was described previously. The transformation is done to provide another set of unique keys, and to allow up to 128 text-records per Note. (Overflow is handled by continuing with the next UID). KEY 1 (Bytes 73-76) is the Note Number of the note. KEY 2 is all Nulls The Note content text starts at Byte 77 of each record. It consists of a number of TLV tagged fields; each tagged field contains one line in the text. When there are multiple text records for a note, the data should be viewed as a continuous string. That is, Bytes 77-nnnn of the second record follow immediately after Bytes 77-nnnn of the first record. CAUTION: The records are often not a full 1024 bytes long. Do not make that assumption. The Tagged Fields containing the text are constructed as follows: Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- NOTES$K_TEXT_STRING 2 02 C2 Variable NOTES$K_TEXT_END 3 03 C3 0 There are zero or more instances of NOTES$K_TEXT_STRING, followed by a single instance of NOTES$K_TEXT_END (with length zero). If there are empty lines, the length is indicated as Hex 00. That is, the coding for an empty line is Hex C200. Summary of TLV Tags Discussed Tag Name Tag Value TYPE LENGTH --- ---- Dec Hex Char Hex ------ --- --- ---- ---- Record 2 NOTES$K_NOTEFILE_TITLE 60 3C < DF3C Variable NOTES$K_NOTEFILE_MODERATOR 54 36 6 DF36 Variable NOTES$K_NOTEFILE_NOTICE 55 37 7 DF37 Variable Member Records NOTES$K_USER_NAME 75 4B K DF4B Variable NOTES$K_USER_NODENAME 78 4E N DF4E Variable NOTES$K_USER_MODERATE 74 4A J DF4A 4 NOTES$K_USER_CREATE_KEYWORD 72 48 H DF48 4 NOTES$K_USER_MAIL_ADDR 73 49 I DF49 Variable NOTES$K_USER_ACCESS_LIST 117 75 u DF75 Variable NOTES$K_USER_WRITE_BYPASS 118 76 v DF76 4 Keyword Records NOTES$K_KEYWORD_NAME 43 2B + DF2B Variable NOTES$K_KEYWORD_X_NOTE 84 54 T DF54 Variable Node Records NOTES$K_NODE_NAME 111 6F o DF6F Variable NOTES$K_NODE_X_USERNAME 112 70 p DF70 Variable Note Header Records NOTES$K_NOTE_AUTHOR 6 06 C6 Variable NOTES$K_NOTE_CREATE_TIME 12 0C CC 8 NOTES$K_NOTE_TITLE 23 17 D7 Variable NOTES$K_NOTE_PEN_NAME 19 13 D3 Variable NOTES$K_NOTE_NUMRECORDS 17 11 D1 4 NOTES$K_NOTE_NUMRESPONSES 18 12 D2 4 NOTES$K_NOTE_WRITELOCK 27 1B DB 4 NOTES$K_NOTE_HIDDEN 13 0D CD 4 NOTES$K_NOTE_NOTEFILE_FILE_NAME 49 31 1 DF31 Variable NOTES$K_NOTE_X_KEYWORD 83 53 S DF53 Variable Note Text Records NOTES$K_TEXT_STRING 2 02 C2 Variable NOTES$K_TEXT_END 3 03 C3 0