From: SMTP%"RELAY-INFO-VAX@CRVAX.SRI.COM" 30-AUG-1994 11:37:55.50 To: EVERHART CC: Subj: Re: pool fragmentation utility available via FTP From: jeh@cmkrnl.com (Jamie Hanrahan, Kernel Mode Systems) X-Newsgroups: comp.os.vms Subject: Re: pool fragmentation utility available via FTP Message-ID: <1994Aug30.004545.4474@cmkrnl> Date: 30 Aug 94 00:45:45 PST Distribution: world Organization: Kernel Mode Systems, San Diego, CA Lines: 93 To: Info-VAX@CRVAX.SRI.COM X-Gateway-Source-Info: USENET In article , stus@pssi.com (Stu Sjouwerman) writes: > In article <1994Aug27.144817.4459@cmkrnl> jeh@cmkrnl.com (Jamie Hanrahan, Kernel Mode Systems) writes: > >>From: jeh@cmkrnl.com (Jamie Hanrahan, Kernel Mode Systems) >>Subject: Re: pool fragmentation utility available via FTP >>Date: 27 Aug 94 14:48:17 PST > >>In article , stus@pssi.com (Stu Sjouwerman) writes: >>> [...] >>> soon and put them in the same FTP directory. The monitor itself is simply >>> looking at he amount of fragments in pool, allocation ratio of memory ( how >>> many times per second is it allocated and deallocated), and calculates how >>> much CPU is spent doing this) > >>By the way, Stu, are you aware that the function you're claiming here is flatly >>impossible to do under pre-6.0 systems? Provided that you want to include >>*all* pool allocations, including those that go to the lookaside lists? (And >>if you don't, you're hardly giving a complete picture of pool allocation.) > > Well, Jamie, it's not so impossible as it may seem to you. and then he says > Obviously one > does not measure the look-aside list allocations, That's what *I* said. (Grin! There actually IS a way to monitor the lookaside list allocations, even though various folks to REMQHI's straight off the lists, without doing nasties like watchpoints, just by intercepting routine calls... I'll be curious to see whether Keith or whomever mentions it.) > What we are concerned with are the _many_ requests that > are _not services out of lookaside lists and wind up with a pool scan. > This is by the way a known problem, and there is even an entry in DSN-link > that explains that is _really_ can be a problem in 6.0 systems. The title is > 'OpenVMS Npagedyn fragmentation .... LISTPREPOP.DAT. I am not disputing that your product can help some systems under some circumstances. (To do so would obviously be a silly position for me to take, as you need only provide *one* counterexample!) However I have yet to see a 5.x system with "thousands of holes in the pool" which wasn't helped by a little twiddling of the various pool parameters, to allow more requests to be satisfied on the lookaside lists. The reason I feel that it is significant that you *aren't* monitoring these highly-optimized paths to the lookaside lists is that your monitoring tools then have no way of telling whether your product would really do any better than simply making the aforementioned parameter changes. I'll also grant you (so that you won't have to post another note to mention it) the advantage of not requiring a reboot to get some of these benefits. Since beginning to follow this thread I logged into DECUServe and looked up the thread on MemoryMaster. There it is stated that in order to "defragment" the pool, MM simply scans the pool looking for uselessly small chunks, and allocates them, taking them out of the free list! (MM also -- so it says in the articles on DECUServe -- periodically checks to see if the pieces it has allocated are next to any freed pieces, ie if by freeing the pieces it owns it could produce larger pieces of free pool, and frees accordingly.) I can certainly envision situations with thousands of little bits of pool ahead of a few decently large ones; simply taking the little bits out of the pool so that the allocator finds the large ones sooner does sound as if it ought to help. This method of "defragmenting" is also a fairly cheap operation. However there are at least two things that encourage me to have further doubts. > By the way, the effects of running the tool can quite easily be measured. The > Kernel mode and Interrupt stack decrease significantly when non-paged pool > gets defragmented, practically always resulting in mode User mode available. Ah, glad you mentioned this. One of my "doubt sources" is the EXTREME difficulty of getting really good before-and-after performance data *when the monitor utility is running on the system that is being measured*. I know; I've done some perf monitor tools. It isn't at all a straightforward problem. (And if it's written by people who think that doing a SETIPL to raise IPL somehow steers your code to execute on the primary CPU, I have *real* doubts about how it can have any validity whatsoever...) The other is that I simply don't like the whole notion of products that are sold with the promise of encouraging the VMS system manager to understand *less* about the system. "Here, don't bother to understand the problem, just throw a little money our way and we'll make it better." Oh, I realize that there's a definite market there -- I've taken support calls from so-called "VMS system managers" who can't even use a text editor, let alone SYSGEN. But I don't have to like it. And I don't. --- Jamie Hanrahan, Kernel Mode Systems, San Diego CA Internet: jeh@cmkrnl.com (JH645) Uucp: uunet!cmkrnl!jeh CompuServe: 74140,2055