From: Jamie Hanrahan [jeh@cmkrnl.com]
Sent: Thursday, August 03, 2000 8:24 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Mapped files/Physical Memory

First, the behavior you're talking about here is properly called PAGING, not
swapping.  NT doesn't really swap in the traditional sense.

That said, Danilo is 99% correct.  Essentially, when you map a file, the
file becomes the backing store for the mapped region of virtual address
space.  It is paged in and out like any other portion of virtual address
space, but instead of being backed up by the paging file it is backed to the
file you mapped.

This is really not a special case.  It's the same way that the various
sections (code, resources, etc.) of .exe's and .dll's get mapped.  e.g.
LoadLibrary doesn't really "load" the library, instead it sets up mappings
between the sections of the DLL and the virtual address space.  When you
then call a routine from the DLL, the routine entry point (and a few more
pages as determined by the readahead mechanism) gets faulted in from the
DLL.

The 1%:  There IS a VirtualLock API.  "When I have to lose a page from my
working set due to my incurring a page fault when I'm already at my limit, I
don't want it to be one of *these* pages."  You can't lock 100 MB at a time
by default, though.  Read the doc on this -- you will have to first increase
your min and max working set with the SetProcessWorkingSetSize API, and that
in itself might be enough.  You still are unlikely to be able to lock 100 MB
on a 128 MB system.  Locking might not help anything anyway, especially if
your pass through the file is one long sequential pass.

It's still amazing that on machine A it took "10 seconds to touch each page"
where on machine B it only took 2 seconds.  No way do page faults take eight
seconds per page!  Perhaps your machine is really tight on memory, such that
faulting in the chunk of file you're looking at pushes the code that's
looking at it out of the working set, and faulting the code in pushes the
data out, and... Try to reduce the number of different pages you're using
over time by ensuring the program exhibits good locality of reference, both
to code and to data.

       --- Jamie Hanrahan, Azius Developer Training
       http://www.azius.com/
       Windows 2000/98/NT Drivers and Developer Training


> -----Original Message-----
> From: bounce-ntdev-2233@lists.osr.com
> [mailto:bounce-ntdev-2233@lists.osr.com]On Behalf Of Danilo Almeida
> Sent: 2000-August-03 Thursday 16:55
> To: NT Developers Interest List
> Cc: NT Developers Interest List
> Subject: [ntdev] RE: Mapped files/Physical Memory
>
>
> On August 3, 2000, Peter Ellis wrote:
> > We want to be able to search the contents of a file very fast.
> > The file(s) is(are) on the order of 100MB. We tried file mapping,
> > mapping the entire file, and this works. But...
> >
> > In our testing we had dramatically different performance on
> > comparable machines except machine A has 128 MB and machine B
> > has 256MB. Machine A took ~ 10s to touch each page where machine
> > B took ~ 2s. There are two camps: a) camp one (me) believes this
> > performance is due to NT swapping; b) camp two believes that file
> > mapping actually reserves physical memory for the file and that
> > once mapped the file resides in memory until unmapped.
> >
> > Questions:
>
> Peter,
>
> I'll try to answer some of these.  I recommend you look at Inside
> Windows NT
> (2nd edition) to get more info.
>
> > 1) Does file mapping reserve all the physical memory required to
> > hold the file or does NT only pre-load some portion of the mapped
> > file?
>
> No.  Basically, the mapped files works just like the rest of
> virtual memory
> with data in it.  As you touch pages, they are paged in (plus
> extra pages as
> determined by NT).
>
> > 2) If file mapping does not reserve all the physical memory required to
> > hold the file is there any Win32 API that will allow allocation of
> > physical memory (outside of writing a kernel mode driver)?
>
> IIRC, you can't force NT to reserve physical memory in user mode.
>
> > 3) What choices do we have to allocate physical memory so that we
> > can load this file into ram and not have to worry about swapping?
>
> I think you'd have to write a driver.
>
> - Danilo
>
>
> ---
> You are currently subscribed to ntdev as: jeh@cmkrnl.com
> To unsubscribe send a blank email to leave-ntdev-247T@lists.osr.com
>
>


---
You are currently subscribed to ntdev as: GlennEverhart@FirstUSA.com
To unsubscribe send a blank email to leave-ntdev-247T@lists.osr.com