Re: Bug report

[Tim Walberg <tewalberg@mediaone.net>]

[-- Attachment #1: Type: text/plain, Size: 5132 bytes --]

"User does not understand how *NIX systems utilize memory" is
not the same thing as "bug". What you are seeing is the kernel
using memory for file system cache.

On 03/22/2001 12:58 -0800, Craig Cummings wrote:
>>	Hi there,
>>	
>>	I think this qualifies as a bug but let me know if this could be a
>>	configuration or hardware issue.
>>	
>>	I've been having problems with memory leaks when I run programs on
>>	large--up to 250MB text files.  (I know this is huge, but that's the 3
>>	billion base human genome for you.)  At first I though it was a Perl
>>	problem but I later found that a completely unrelated C program also
>>	caused memory leaks.  I recently upgraded to the 2.4 kernel, hoping to
>>	solve these problems (see below).  However, the memory leaks are still
>>	happening and this time I know the problem is at a deeper level than my
>>	programs.  Some standard UNIX programs are leaking a lot of memory.  I
>>	would appreciate some advice and ultimately, a fix.  Unfortunately, My
>>	programming skills are not sufficient for tinkering with the kernel
>>	source.  Thank you, in advance for your help.  Details follow.
>>	
>>	Regards,
>>	
>>	Craig Cummings
>>	
>>	
>>	Here are the specs for my system:
>>		Dell Precision XPSt700, Pentium III, 512 MB RAM
>>		I've recently upgraded from Red Hat 6.2 with the 2.2.14 kernel to
>>		Red Hat 7, then built the 2.4.2 kernel on my own.
>>	
>>	Here's what happens with grep:
>>	
>>	Output of free, freshly booted system:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616      47516     466100          0       2476      27048
>>	-/+ buffers/cache:      17992     495624
>>	Swap:       128480          0     128480
>>	
>>	Output of free after grep 'NT_005289' Data/hs_chr12.fa:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616     183548     330068          0       2624     159616
>>	-/+ buffers/cache:      21308     492308
>>	Swap:       128480          0     128480
>>	
>>	Output of grep 'NT_005289' Data/hs_chr2.fa:
>>	
>>	>gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working draft sequence segment
>>	
>>	Output of free after this grep:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616     424272      89344          0       2860     394232
>>	-/+ buffers/cache:      27180     486436
>>	Swap:       128480          0     128480
>>	
>>	Output of grep 'NT_005289' Data/hs_chr2.fa:
>>	
>>	>gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working draft sequence segment
>>	
>>	Output of free after this grep:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616     424272      89344          0       2860     394232
>>	-/+ buffers/cache:      27180     486436
>>	Swap:       128480          0     128480
>>	
>>	File sizes of the two files grep'ed:
>>	
>>	-rw-rw-r--    1 cummings genomics 135744469 Mar 12 22:09 Data/hs_chr12.fa
>>	-rw-rw-r--    1 cummings genomics 240244039 Mar 12 22:24 Data/hs_chr2.fa
>>	
>>	Note that these file sizes are equivalent to the amount of memory leaked
>>	when grep is called on that file.
>>	
>>	When I grep the same file a second time, very little additional memory is
>>	leaked.
>>	
>>	
>>	This same phenomenon occurs when I run a different UNIX program, e.g. wc:
>>	
>>	Output of wc -l Data/hs_chr3.fa:
>>	
>>	2915465 Data/hs_chr3.fa
>>	
>>	Output of free:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616     511520       2096          0       1252     481020
>>	-/+ buffers/cache:      29248     484368
>>	Swap:       128480          0     128480
>>	
>>	Interestingly, after running wc a second time on the same file, it goes
>>	very fast and very little additional memory is leaked:
>>	
>>	             total       used       free     shared    buffers     cached
>>	Mem:        513616     510732       2884          0       1204     480948
>>	-/+ buffers/cache:      28580     485036
>>	Swap:       128480         40     128440
>>	
>>	
>>	-------------------------------------------
>>	Craig Cummings, Ph.D.
>>	
>>	Relman Laboratory
>>	Stanford University School of Medicine
>>	Department of Microbiology and Immunology
>>	
>>	e-mail: cummings@cmgm.stanford.edu
>>	phone:  650-498-5998
>>	fax:    650-852-3291
>>	
>>	-
>>	To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>	the body of a message to majordomo@vger.kernel.org
>>	More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>	Please read the FAQ at  http://www.tux.org/lkml/
End of included message



-- 
+--------------------------+------------------------------+
| Tim Walberg              | tewalberg@mediaone.net       |
| 828 Marshall Ct.         | www.concentric.net/~twalberg |
| Palatine, IL 60074       |                              |
+--------------------------+------------------------------+

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]