From:	SMTP%"everhart@star.zko.dec.com" 10-JUN-1996 18:13:42.12
To:	EVERHART
CC:	
Subj:	fastio.txt

Date: Mon, 10 Jun 1996 16:30:52 -0400
Message-Id: <96061016305211@star.zko.dec.com>
From: everhart@star.zko.dec.com (Glenn C. Everhart 603 881 1497)
To: everhart@gce.com
Subject: fastio.txt
X-VMS-To: SMTP%"everhart@gce.com"

From:	STAR::MSHERLOCK    "Margie Sherlock" 22-AUG-1995 15:35:44.76
To:	STAR::EVERHART
CC:	MS
Subj:	Joh Hallyburton's Fast-IO driver info....................

From:	EVMS::HALLYB       "John C. Hallyburton, Jr."  7-FEB-1995 09:38:05.92
To:	STAR::MSHERLOCK
CC:	
Subj:	Fast-IO driver doc of sorts.

This is ill-organized and poorly put together. But it does have the virtue
of containing most if not all of what I think needs to go into the driver
manual, and any other "Internals/Update" material.

Feel free to come to me with any questions.

  John


		Fast-IO and foreign drivers

- Overview

Fast-IO System service operation is functionally similar to $QIO in that
$IO_PERFORM[W] prepares an IRP and calls a driver FDT entry point. When
the driver FDT routine returns, EXE_STD$QIODRVPKT is called to start the
I/O in the usual manner. Alternatively the FDT routine can finish an I/O
via EXE$FINISHIO or abort an I/O via EXE$ABORTIO, similar to $QIO.

Fast-IO performance benefits come from a number of "streamlining" steps.
For example, an IRP is permanently allocated by $IO_SETUP and used repeatedly
by $IO_PERFORM. But the largest benefits stem from having user buffers and 
the I/O status area permanently locked down and mapped via system space. 
This means:

  * for direct I/O, no per-I/O buffer lock/unlock required.
  * for buffered I/O, no separate system buffer required since the 
    user buffer is always addressible.
  * Fast-I/O operations can complete at IPL 8, avoiding the interrupt chaining
    usually required by the more general-purpose $QIO system service. 
    This saves CPU time but also restricts the types of operations that 
    can be performed.

Fast-IO System Services support the four operations IO$_READVBLK,
IO$_WRITEVBLK, IO$_READLBLK and IO$_WRITELBLK. All others are rejected
by $IO_SETUP.

Fast-IO will work with any driver provided the driver needs only the 
P1, P2 and P3 arguments.

In order to obtain maximum performance, drivers will need to write a 
FAST_FDT routine. Drivers without a FAST_FDT routine will still run 
with Fast-IO system service calls, but performance gains are minor.
In any event driver writers must inspect their drivers for any of the 
required changes listed in section <RC>.

- IRP usage

The following changes describe ways Fast-IO deals with IRPs.

IRP$L_STS:
* IRP$V_FASTIO is set to indicate this is a Fast-IO IRP
* IRP$V_FAST_FINISH is set when the I/O is suitable for IPL8 completion
* IRP$V_FASTIO_DONE is set upon completion of the I/O operation, regardless
  of the setting of IRP$V_FAST_FINISH

IRP$L_RMOD:
* ACB$V_NODELETE is set since the IRP is not to be deleted upon AST delivery
* ACB$V_FLAGS_VALID is set to indicate extended flags are in use
* ACB$V_QUOTA is not set, even if an AST is requested

IRP$L_FLAGS:
* ACB64$V_FASTIO is set to indicate to ASTDEL this is a Fast-IO ACB/IRP;
  ASTDEL cannot examine IRP$L_STS as that bit is not part of an ACB or ACB64

IRP$L_SVAPTE:
* Set to the Buffer Object Descriptor block (BOD) for the Buffer Object for
  the caller's data buffer. This is important for drivers that do direct I/O.
  Drivers will need to replace this value with something appropriate for the
  particular operation.

IRP$PS_CCB:
* Contains a valid system space address of the Channel Control Block (CCB)
  for the I/O channel specified by the $IO_PERFORM[W] call. The CCB can be
  accessed from either kernel mode process or system context.

IRP$PS_IOSB:
* Contains a valid system space address of the I/O Status Area (IOSA) as
  specified by the $IO_PERFORM[W] call. The IOSA can be accessed from either 
  kernel mode process or system context.

IRP$Q_QIO_P1
* Contains the $IO_PERFORM "bufadr" parameter. If a FAST_FDT routine is
  called, the address is converted to a system-space equivalent using the
  information in the BOD for the data buffer.

IRP$Q_QIO_P2
* Contains the $IO_PERFORM "buflen" parameter.

IRP$Q_QIO_P3
* Contains the $IO_PERFORM "media" parameter.

IRP$Q_QIO_P4, IRP$Q_QIO_P5, IRP$Q_QIO_P6
* Zeroed.

- The I/O status area (IOSA)

An IOSA is the Fast-IO analog of the IOSB used by $QIO and various other
system services. IOSA offsets are defined in STARLET module IOSADEF.
Fast-IO provides drivers with a driver-specific longword, IOSA$L_RESD,
for use as the driver sees fit. IOSA$L_RESD is neither read nor written
by Fast-IO system services.

Unlike $QIO, Fast-IO requires the caller to supply an IOSA and
requires the IOSA to be part of a buffer object. The system virtual
address of the IOSA is stored in the IRP at offset IRP$L_IOSB.

- FAST_FDT routines and other driver changes to support them

After $IO_PERFORM prepares an IRP it looks in the driver dispatch table
(DDT) at entry DDT$PS_FAST_FDT. If nonzero, $IO_PERFORM calls the entry
point with the same arguments as a standard FDT routine. Note however
that there is only one FAST_FDT routine defined which must serve all
4 valid function codes, unlike regular FDT dispatching which defines
one FDT routine per function code.

Support for function modifiers is decided by the driver FAST_FDT
routine. If a driver detects an unsupported modifier it should call
EXE$ABORTIO with status code SS$_ILLMODIFIER.

The FAST_FDT routine will be dealing with buffer object addresses for the
caller's I/O buffer and IOSA. These addresses have several properties
worth noting:

* These are system-space addresses and are always valid. Thus they may be
  read or written an any time, including from elevated IPL. This includes
  the range of (contents of IRP$Q_QIO_P2) bytes, starting at the address 
  in IRP$Q_QIO_P1, and the IOSA$K_LENGTH bytes starting at the address in 
  IRP$L_IOSB.

* These are guaranteed to belong to the caller's address space at all times.
  This means they do not need to be PROBEd before read or write. In fact, it
  would be an error to PROBE them because the caller would not normally have
  access via the system space addresses since those are mapped in Kernel mode.
  For this reason the FAST_FDT routine is passed the \system space/ VA in
  IRP$Q_QIO_P1 but the \process space/ VA is passed to the (non-FAST_) FDT 
  routine if there is no FAST_FDT routine.

* For the data buffer, drivers can convert between system and process VA 
  using the BOD passed in IRP$L_SVAPTE. The calling process does not know
  and cannot use the system space VA so drivers must be careful to translate
  any addresses to process space before giving them to the caller.

* Drivers doing direct I/O reads must set the M bit in the \process space/ 
  PTE so the page will be cleaned up properly. (I think.) HOWEVER, the 
  Direct I/O Buffer Map should use the system space PTEs since they can
  be used without additional locking. Doing I/O through process space PTEs
  would require locking down multiple levels of PTEs, overhead that Fast-IO
  is designed to avoid.

The FAST_FDT routine should be aware that:

* The IOSB will be quadword aligned and the buffer in IRP$Q_QIO_P1 will be
  aligned to 512 bytes. 

* IRP$Q_QIO_P2 may contain any value satisfying 0 <= IRP$Q_QIO_P2 <= 
  (upper addressing limit of the data buffer object) - value in IRP$Q_QIO_P1

* IRP$Q_QIO_P3 may contain any value and is passed to the driver unmodified.
  If drivers want to use IRP$Q_QIO_P3 as a pointer they will not be able to 
  make accessibility or alignment assumptions.

Drivers wishing to do buffered I/O may wish to take advantage of buffer
object addressibility and read or write the caller's buffer from within
the driver. If the driver completes a buffered read operation, i.e.,
copies data to the user's buffer, it should zero out IRP$L_SVAPTE, delete 
the corresponding BUFIO packet, and maybe return any JIB bytecount charged
(typically the value in IRP$L_BOFF). This will prevent VMS from copying
the data redundantly.

- Fast-finish

Fast-finish means the I/O is completed at IPL8 without an IPL4 or IPL2 Special
Kernel AST, presuming the following:

  * IRP$V_FAST_FINISH is set in the IRP. This is the case when Fast-IO calls
    FAST_FDT routines if certain minimal standards are met. The FAST_FDT
    routine may clear this bit if IPL8 completion is inappropriate.

  * Driver calls one of IOC$REQCOM, IOC$ALTREQCOM or COM$POST (or any of
    the _STD variants) at IPL8 to complete the I/O.

  * The I/O completes with good status.

Drivers may wish to clear IRP$V_FAST_FINISH if they believe an I/O should not
be completed directly at IPL8. This would happen in the case of a segmented
virtual I/O, where IPL4 is used to detect this situation and queue the next
segment.

- <RC> Required driver changes

In order to work with Fast-IO, with or without a FAST_FDT routine, drivers
must obey the following rules if given a Fast-IO IRP:

1. No changing PCB$L_DIOCNT or PCB$L_BIOCNT. Fast-IO requests do not modify
   these fields. Typically the right thing to do is change the driver to
   branch around code that modifies these fields if IRP$V_FASTIO is set.

2. There is a subtle difference in the use of ACB$V_QUOTA in IRP$B_RMOD.
   Drivers do not normally care about this but it is a change in behavior.
   Normally if a user AST is requested then ACB$V_QUOTA is set and
   PCB$L_ASTCNT is decremented. Fast-IO does not charge for ASTs, so
   drivers should not rely on ACB$V_QUOTA to say if an AST was requested;
   instead they should check IRP$L_AST for nonzero value. And, as with the
   PCB I/O counts, drivers should never change PCB$L_ASTCNT for Fast-IO IRPs.

3. The IOSA pointed to by IRP$L_IOSB is a system-space address accessible 
   only from Kernel mode. Drivers do not normally access this structure, but
   any who do should be aware that:

   * IRP$L_IOSB points to an IOSA (not an IOSB) if IRP$V_FASTIO is set
   * The SVA in IRP$L_IOSB is not accessible from outer modes, so any
     kind of PROBE vs. caller's mode is likely to fail. If IRP$V_FASTIO
     is set, do not PROBE, just use the address in IRP$L_IOSB.

4. Drivers that think they know how IPL4 I/O posting works should not try
   to "complete" Fast-IO IRPs. Better to post them to the IPL4 queue.