From: SMTP%"leichter@lrw.com" 9-JAN-1995 16:52:32.52 To: EVERHART CC: Subj: re: Re: Tracking down C memory leak From: Jerry Leichter X-Newsgroups: comp.os.vms Subject: re: Re: Tracking down C memory leak Message-ID: <9501091711.AA14028@uu3.psi.com> Date: Mon, 9 Jan 95 11:31:08 EDT Organization: Info-Vax<==>Comp.Os.Vms Gateway X-Gateway-Source-Info: Mailing List Lines: 44 To: Info-VAX@Mvb.Saic.Com Since several people in this thread have commented that they had programs that "ran just fine" under Unix, failed (at memory without bound) with the VAX C memory allocator, but worked fine with the VAXC$xALLOC_OPT routines, it's worth looking at the history here to understand why. The old Unix malloc() algorithm, as a side-effect of its operation, allowed you to realloc() a block of memory that you had just free()'d. Not only did this work, but it caused the memory allocator to coallesce blocks on its free list, which could improve performance. While these facts were never documented (ha ha), they were used in a number of Unix programs. To support those programs, VAX C's malloc(), which was built on top of LIB$GET_VM, added a small hack to ensure that this programming style would work. (Basically, it kept the most recently freed block of memory "in reserve", not handing it back to LIB$GET_VM until the next free() call.) This little hack has all sorts of unfortunate side-effects, as has been pointed out. The original Unix malloc() algorithm was designed on a 16-bit non-paged machine. It had horrible performance on paged machines with much larger memory spaces, and was long ago replaced by better algorithms. These algorithms didn't need the old hack for compacting the free memory list, and in fact no longer supported the "realloc() of free()'ed memory" trick. Unix programs were gradually changed not to rely on the internals of the malloc() implementation, but rather on its documented interface. (What a concept.) Still, VAX C, in the name of backward compatibility, retained its little hack. The VAXC$xALLOC_OPT routines are, in fact, completely compatible with all contemporary (last 15 years?) Unixes, and there is no good reason not to use them; but due to concern for upward compatibility, they remained "optional", rather then replacing the old defaults. In general, malloc()/free() are a poor choice for high performance programs that do a lot of dynamic memory management. The implementations are different Unix systems vary from quite good to horrible. If you really need to worry about the time and memory your program needs, a special-purpose allocator, probably using look-aside lists for common sizes, will win just about every time. While I rarely actually replace malloc()/free(), I almost never call them directly; I define one or more private memory allocators (more than one if I have clearly distinct uses for the memory), and simply have them call malloc() and free(). Should they ever become bottlenecks, I can quickly replace them. (Actually, I generally have these routines check for a NULL return from malloc() and abort the program. Very few programs are written to survive failure to get enough memory, and it's better to get a message saying that than a message about an access violation later, when the program tries to use the NULL pointer.) -- Jerry