From:	MERC::"uunet!CRVAX.SRI.COM!RELAY-INFO-VAX" 29-JAN-1993 03:02:49.10
To:	info-vax@kl.sri.com
CC:	
Subj:	Re: Image activations, page faults and disk I/O

In article <1993Jan26.155208.131143@mswe.dnet.ms.philips.nl>, hdavids@mswe.dnet.ms.philips.nl (Henk Davids) writes:
> Image activations under VMS are expensive. What's worse is that the
> repeated activation of the same image by the same process does not get
> cheaper on successive calls. 

Yes it does, if the image is installed /shared, and if the shareable pages
defined by the image are still on the free page list. 

> Likewise, multiple activations of a
> particular image by many processes are probably just as expensive.

No.  If the image is installed shared, you get "global valid faults" when
multiple processes map the same (shareable) page within the image.  These are
soft faults. 

>  But the actual code pages of the image still have to be read from disk.

only if they're not on the free list, and also not in someone else's working
set. 

> When you INSTALL an entry in the Known Files list with the /SHARE
> qualifier, a global section will be created for every pure code
> section in the image file.  

Actually a global section is created for *every* section (not just "pure code",
by which I assume you mean "shareable, read-only").  The "impure data" image
sections become writable, copy-on-reference global sections.  The *definition*
of the section is still "global" even though the pages it defines won't be
shared.  (This is a subtle point which just about everyone gets wrong...)

> Presumably, the image activator should
> then map process pages to these global sections. However, that does
> not explain all the disk accesses that I see. 

Well, I see fewer "pages read" from MONITOR PAGE when I install an image.  Bear
in mind that almost every image has some private, writable, non-demand-zero 
code sections.  For instance, storage allocated by .ADDRESS and .ASCID 
directives, which have to be fixed up at activation time, and which therefore
can't be shared.  In a non-trivial image you will never get rid of all the
reads from the image file.  

> So, the question is: does the image activator in this release of VMS really
> use these global sections? 

Yes, it really does.  This is easy to demonstrate (at least on a system with
enough memory for it to do any good):  Use MONITOR PAGE to watch the number and
types of page faults and see what happens with, say, the DIRECTORY command
before and after you install it /shared.  (I pick on DIRECTORY.EXE because it
doesn't need to be installed with privileges.)

> At image rundown, the pages that were mapped to shareable, read-only sections
> are moved to the free list.  Normally process pages that are added to the 
> free list can be reused, if they have not yet been used for something else. I
> suspect that image rundown removes the connection between the process
> and the image pages 

It removes the connection between the process and the image, yes.   But
shareable, read-only pages released to the free list are moved to the *end* of
the free list, where they won't get reused soon. (And, at image rundown,
copy-on-reference pages are released to the *top* of the free list, so they get
reused before the "maybe still useful to someone" pages that are nearer the
end.)  The connection between the global section and these pages persists until
the pages reach the top of the free list and are used for something else. 

If the shared (or shareable) image is activated again before the pages are
used for something else, faults to these pages are resolved simply by pulling
those pages back off the free page list.  

> because even on an empty VMS system the hard page faults keep the disk busy. 

Copy-on-reference pages being faulted in.  

> Why would image rundown go to such trouble? [removing the 
> connection between the process and the image pages]

First because it isn't much trouble; and second because it *has* to.  Remember
that the connection is via the process page table entries.  These correspond
one-for-one with pages in virtual address space.  The next image you run is
going to use the *same* virtual address space (normally starting at 200 hex),
and hence will use the same PTEs, so you can't possibly keep pointers to a
previous image's pages in them. 

Again, once a global, shareable page is released from everyone's working set,
the connection between the *global* page table entry (which assists in 
resolving faults to pages defined by global sections) and the physical page
on the free page list *is* retained.  

> Another suggestion: failing a general solution, perhaps DEC could
> provide a DCL image in which al lot of these images have been combined
> into one in future versions of VMS, for those of us that have plenty
> of memory, but not that much bandwidth for page faults? I'm thinking
> especially of popular images like SET, DIRECTORY, COPY, RENAME, DELETE
> etc.

I don't see the point of this.  Images don't get faulted in all at once. Let's
say that one person is doing a DIRECTORY, another a COPY, and another a DELETE.

 - If they're separate images,  each process faults in the image-specific
stuff (different for each process) and the common RTL routines (shared). 

 - If they're the same image, each process faults in the function-specific
stuff (different for each process) and the common RTL routines (shared). 

What's the difference?  

The only difference you'd see in the behavior would be fewer global section
table entries used.  Possibly. 

> Yet another puzzle: why would image rundown have to deallocate all
> your pages in P0 space? Couldn't it just leave the working-set intact,
> so that multiple activations of the same image could re-use what was
> left over?

There are two reasons.  

One:  A working set list entry is meaningless without a corresponding virtual
page number.  In fact, the high bits of the WSLE *are* the virtual page number.
This is necessary because you have to be able to find the virtual page number
(and from there, the address of the corresponding process page table entry)
from a working set list entry.  Now, the same virtual addresses that the "old"
image used are going to be re-used for the next image.  Hence there's no sense
in keeping old working set list entries around.  A physical page can't be part
of a process working set unless the image the process is running defines the 
contents and attributes of the page.  

Two:  It wouldn't be any better than what VMS does now! Remember, the pages
that can be shared are already being cached at the end of the free page list,
and faults to these can be resolved quickly.... unless they "boil" up to the
top of the FPL and get re-used for something else.  

I suspect that what you're looking for is a way to prevent them from being
re-used for something else. (This is essentially what TURBO does.)  That's
fine, if you have enough memory to hold all the shareable read-only pages for
every image that is run on the system!  In most systems, though, you are going
to want to re-use those pages eventually:  You want the pages defined by images
that are activated often to stay near the bottom of the FPL so they probably
won't get re-used... 

...and that's exactly what the existing mechanism does. 

If you aren't seeing a reduction in "page reads" and an increase in "free page
list faults" and "global valid faults" from installing often-used images
/shared, you probably don't have enough spare memory to allow the mechanism to
work effectively.  

> Suggestions, solutions (and flames :) are welcome.

I suggest you read the "paging dynamics" and "image activation and rundown"
descriptions in the IDSM again.  

	--- Jamie Hanrahan, Kernel Mode Systems, San Diego CA
drivers, internals, networks, applications, and training for VMS and Windows NT
uucp 'g' protocol guru and release coordinator, VMSnet (DECUS uucp) W.G., and 
Chair, Programming and Internals Working Group, U.S. DECUS VMS Systems SIG 
Internet:  jeh@cmkrnl.com, or hanrahan@eisner.decus.org  Uucp: uunet!cmkrnl!jeh