There are a number of techniques which don't get discussed a lot in
DECUS circles. Sometimes I wonder whether this happens because they
are too valuable and get used in commercial packages.
  Anyhow, having figured out some of them I propose to write a bit
about them here.
(    Remember that the art of system programming is the use of
mostly defined internal interfaces for other purposes. OCCASIONALLY
one has to add patches at not so well-isolated spots, but that is
bad practice generally because it is very vulnerable to small
changes in a base OS messing them up...even automatic recognition
of a code sequence is not a guarantee all will be well.)
   In the main the ones I know of are techniques for stealing
information flows. You see these crudely in vddriver and
fddriver, where the drivers mess with IRPs and call other
parts of VMS from inside start-IO packets. (The old technique
in vddriver of starting in one set of FDT entries and switching
cleanly to another is also useful, though my drivers don't use it
anymore.) The scheme in fddriver to have a process do all real
work in doing disk I/O was also cute. Here the only real trick
was completing two I/O's at one time.
   One can go far by stealing IRP completions. You do this
by replacing irp$l_pid with a system address; it will be called
via JSB from the I/O post processing (ipl 4) processing. You
can of course replace the old irp$l_pid and recall the
completion, or use this as a way of calling a driver as if it
were a subroutine. Consider the merits of stealing the FDT entries
for the ctdriver in order to monitor what it's doing. You can
modify the IRP for a read to come back to your kernel code after
the read finishes, let the driver do its' thing, and then after
grabbing off a copy of the data go and replace the IRP field and
complete the I/O yourself (reqcom or the like). By grabbing
access at FDT time, one can capture data written to any device,
and by grabbing the IRP and (perhaps arranging a special ast
prior to the rest of i/o completion stuff) one can grab
the data read from the device then. It'll work for any
device using the normal driver QIO$ interface. It will work
less well than stealing port-class communications for some
cases (since it requires waiting for each I/O to complete), and
where there are alternate paths into a driver these will not
be seen (true in some networking drivers including DECnet at
low level), but for process I/O it is not too shabby. The ability
to wiretap arbitrary I/O streams has many uses. (Not ALL of them
are for nefarious purposes!)
  This kind of thing can monitor any device, so it could for
instance be used to journal all writes to a disk also. When
doing that, capturing the IRP should be unnecessary; once
a copy of the data is made, the I/O can go on normally.
   Other FDT stealing tricks are clear. Suppose you want to
extend the disk filesystem so files can be migrated around.
You can steal the FDT entries for a disk driver so that you
get control in your code on an open before the real DEC FDT
code does. You can queue a message to a daemon process then which
will do stuff like decompress or copy a file to the disk (if the
fdt is not called by the daemon of course), and it can then
send a special kernel ast to the original process which can be
made to start the DEC FDT code which will then find a normal
file. If your process finds the file someplace it wants to leave
it, all it needs to do is be willing to handle virtual I/O itself
(and get control on virtual I/O) and it can leave a file on
secondary storage. You can mark files with application ACLs to
store real locations on backing store, mark compress methods,
or the like, without screwing around with unused header info.
Once in the FDT routines, the IRP already exists and the
user mode code is already waiting if it likes for I/O
completion; no further synchronization of that is needed.
This amounts to allowing two (or more) ACPs per disk. Neat
stuff is possible with tricks like that.
  Remember of course that FDTs already have a process context
(that of the requesting process) and so a second ACP might
be implemented more like a new part of an XQP. Things like
decompression are best done by a daemon, but a process can
be waited, and can do I/O. I've thought about a remote
virtual disk that does r/w access from many machines, using
the driver to do file operations and moving the FDT procesing
and virtual i/o across nodes. However, if one wants to set
up window blocks like ods2 does, it may be possible to
avoid acp/xqp interference for EVERY read/write, though some
networking code must be there to move the data. This kind
of thing can be used, for example, for nfs access where
a volume at the other end does exist and you can arrange
your network data mover code to recognize that r/w to certain
blocks maps to r/w to certain remote file handles or even
remote blocks.
  You can also at FDT time arrange to steal write-virtual and
log writes to a channel or channels. (Best to keep your own
table per process...use a logical to hold the root to this
perhaps). Also at FDT open time other things can be added.
For example it's possible to check image name and perhaps
allow opens only by certain images of certain files. Or limit
the files a given image may write to certain ones. This might have
much the same effect as giving images identifiers, but stays out
of the adding identifiers business. This avoids problems at image
rundown, since that can take place so many ways.
   The irp$l_pid hook is in there and heavily used by MSCP code.
It is unlikely to go away.
   If you want a driver to work transparently with MSCP you can
of course structure it as a port driver rather than a normal
driver as vddriver and fddriver are. The access MUST still be
interlocked across the cluster though. A port driver buys very
little except getting mscp requests direct instead of needing
the mscp server. It still has to know how to interpret mscp
requests, and if it's a virtual device will still have to have
that cpu do its' access. Since SCS services are not well documented
for the world by DEC, this is not so simple.
   In fact handling cache and similar cluster stuff should be done
via locks (cluster wide) so that writes get locked via the distrib.
lock mgr. and drivers understand this. The SCS code is not readily
available, so finding out how DEC uses locks is hard, but
one can add one's own locking structure readily enough. Block
oriented caches will probably need such a thing in any case.
   Then again by stealing a start_io entry point it's possible to
do things like disk caching. The tricky part is handling port
drivers as well as local ones. They also have a start_io vector
but its' format differs from the normal driver's since they use
the class driver. Similar stuff can be done for tape of course.
Paul Sorensen's CDDRIVER is an example of a caching system, but not for port
drivers. Port driver i/o needs to be locked clusterwide anyhow
since port drivers don't have a clue about cluster synch. (I
finally got a look at some). Grabbing start-io gets the local
part of the caching done but the cluster synch. needs locks
to be added explicitly to ensure that a write to a cached
block owns it exclusively. I'd be inclined to have locks on a section
of the cache, I think, so as not to proliferate too many locks.
Converting locks is faster than creating them...
   The thing to be sure of is that whenever a cached block is to
be written, it must be owned exclusively by the processor that
will do the write, invalidated anywhere else, for whatever
purpose.
	Another use of stealing FDTs can be enabling protected
subsystems. That is, by catching open and close on a file, one
can use $grantid and $revokid on open/close respectively and
then any app that opens a file will gain the granted identifiers
for the duration of that file open period. Images, it should be
recalled, are open during execution. If a count of grant/revoke
can be maintained in process context, it will make the operation
easier to use; a logical in the process table might be a place
to keep such a thing per identifier. This would be rather more
flexible and useful than just being able to install images with
identifiers; one could for instance say that a sensitive data
file had identifiers, and anyone accessing it would then get
some identifier that might allow access to other related data
but at the same time prohibit access outside.
	The identifier scheme has a weakness in that it operates
at open time, of course; this is a problem. One could perhaps
add some further checks when these revokid/grantid's were done
and disable access by hand to devices, but it might be better
just to accept the limitation in the interest of not screwing
performance up too badly. This needs more thought.

	Glenn Everhart
	1/6/92