To: S. Noyes (and anyone else interested)
From: Glenn Everhart
Date: 21-December-1995

Re: Ways VMS can be made more interesting to application developers

There are a few ideas I've had over time that could be of some interest
if implemented within VMS. What I'll describe I have partially implemented
in the past, but for specific purposes. I believe that a more general
implementation could provide a set of APIs that would be highly useful
in bringing some new features to VMS.

I. The event registry. The notion here is that it is possible to arrange
a selectable intercept on various VMS operations and allow apps to
notify the registry that they want to know about those operations, either
before they happen or after. This can be done for file operations
rather easily and selectably. Likewise it can be implemented by pieces
of system code that find out things like pool getting low, and similar
techniques could be used to flag when, say, logins occur, or whatever
else one felt to be interesting.

For a third party to build such a thing makes sense only for special
purposes. For a couple examples:
1. I built an undelete facility. It works by intercepting delete and
notifying a server beforehand so the file can be saved and the delete
possibly faked (so rename works). It also monitors create and modify so
it can tell when a volume is running out of space and can do volume
cleanup then, if needed. Because it gets access before the operation,
it can perform (from user mode!) whatever operations the site likes, and
allows undelete of whatever was deleted and not yet trashed. It also
has facilities to store deleted files automatically on backup if such
is wanted.
2. I built an extended security facility. It gets a server notified when
open or delete is tried (before the actual operation) and can examine
username, privs used, image used, where the user is, time of day, and
optionally the presence of a password for the file to decide whether to
grant access, and what kind of access to grant (r/w or r/o). Because
it runs before the operation, it can reliably prevent deletion. It also
can silently open a different file if access is to be denied, so an
intruder can't tell the system is on to him, and it has some privilege
control and anti-Trojan features.
3. I built an HSM package, again monitoring open and allowing transparent
file inswap or linkage.


These features illustrate the security usefulness of the kind of event
registry I am mentioning. However, the same features can be used for
hot-links as are common in PC space. That is, an app can tell the
registry to notify it of mods to some class of files (which may be a
very large class, unfeasible to just take out locks on every member)
and get notified without having to poll. Thus (as an example) it becomes
simple to build large summary spreadsheets whose cells contain references
to data in other files, knowing that when any of the subordinate files
were touched, the master spreadsheet would be notified and be able to
recalculate their values. A simple query of the form "tell me about
any writes to files named "*.DBM" would do the trick if this were
a way to tell that class of files apart from others.

Fullscreen directory maintenance programs in VMS now have to poll
to see when directory contents change. With a registry of the form I
am mentioning such polling could be avoided.

The more events became available through such a registry the more various
sorts of monitoring applications now somewhat common in PC space could
be built. The nice thing VMS would offer is the ability to have pretty
much as many of them at a time as one wanted. To an applications designer
this is a very attractive set of primitive capabilities. To date we don't
get so many of these because those who might build the capabilities on
top need to be able to build the underlying engines.

The VMS audit facility provides some of these event notices via its
listener mailbox facility, but is only after the fact (the bane of
security folks...all you can do is delete the process or $forcex
the image, and pray you got to do it before damage was done). I'm
arguing for being able to do a wider selection range, and before OR
after the fact.

II. File open restart

I implemented partial softlinks in VMS by allowing a tag to be added
to a file that contained a device and a file ID. A kernel thread read
the tags when open was being tried, and if such a tag was present, I
would reset the channel UCB and the open IRP to the new device,
reset the open to get the file ID pointed at, and let the open proceed
to get the new file. (A close monitor let me reset the channel to the
original one on close.) 

This works nicely, but is not recursive.

It would be quite a bit more useful to implement an open-restart
capability such that a file open would be able to be sent to
a decision server able to respond with a "restart" code. The
effect of such an operation would be that the open is restarted
for a new device (and possibly by computing a new path to the device)
and then allowed to go through the entire cycle again. This would
make it simple to have softlinks just like unix has to turn
several filesystems into one effective larger one.

It is also possible to use this sort of primitive to build a system
where one does I/O to volumes by label, instead of to devices as
now. An open to a file on disk$foo: might come to a decision server
and discover that disk$foo wasn't mounted anywhere. This could give
rise to a request to mount disk$foo somewhere and a restart of the
operation when it was. The idea would be like what one sees with
a single floppy on an msdos box being able to act as drive A: and
drive B: at one time. (This kind of indirection via a server can
easily be made to happen only when a disk is not mounted.)

A restart capability is also a simpler way to express what a HSM package
has to do, and can make it much more general. It might be able also
to allow systems to be set up that effectively merge many disks
into one larger facility without forcing them to be backed up all
at once or being all at risk if any of them fail. In that space
it'd be only one option, but I've heard requests for something
of the kind. Also I've heard of folks wanting to make their whole
disk farms appear to be one file structure instead of a bunch of
separate ones.

The volume based filesystem access is however something people seem
to want.


These functions can be implemented via i/o intercept techniques.
I've used one (and published the code) that is device specific and
allows me to control what gets monitored (so there'd be no delete
monitoring, for example, unless someone wanted it, and only on those
devices needing it). They are not, in their general form, sensible
for ANY 3rd party to write. Selling them would get too hard, regardless
how useful they might be, since a 3rd party wanting to use such a
facility must deal with the other "3rd party" as well as the OS vendor.

Having facilities of this kind could make VMS considerably more
interesting for 3rd party developers, could be useful in moving
NT capabilities to VMS, and might provide incidental means for Digital
to add some useful features. (The undelete I mentioned is a feature
everyone needs, and the kind of security facilities one gets very
inexpensively with this kind of infrastructure can make mainframe
security look pretty sick, which isn't a bad statement to be able to
make either.)

I don't know who is the best person to describe this to, but will be
happy to talk about these notions with whoever wants to talk about
them. I do believe the kind of API I'm talking about can make VMS
a better system.