Memory leak/OOM in 2.2.11

Ilpo Ruotsalainen (lonewolf@cs.hut.fi)
Thu, 12 Aug 1999 21:50:31 +0300 (EET DST)


This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.

---559023410-851401618-934483831=:1744
Content-Type: TEXT/PLAIN; charset=US-ASCII

Note: Sorry if this is already known problem. I am not on the linux-kernel
mailing list, please Cc replies directly to me.

2.2.11 seems to be leaking memory somewhere (probably in the networking code)
for me. Seems to be 100% reproducible.

Running my server program (described later) on my dual P3 running 2.2.11
and bombing it with requests from 2 Sparcstations running Solaris 2.7
(each running 30 clients in parallel) for a while my machine seems to run
out of memory in kernel-space and OOM-kills every process which tries to
run. Situation doesn't get any better even when I kill the clients on the
Suns, only thing I can do is cold-reboot the machine. Kernel doesn't lock
up nor panic nor give any error messages.

Shift-scroll lock output from one of the "crashes":

Mem-info:
Free pages: 132kB
( Free: 33 (256 512 768)
21*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB ...
2kB)
Swap cache: add 2398, delete 2398, find 11...
Free swap: 258752kB
65520 pages of RAM
1098 reserved pages
193 pages shared
0 pages swap cached
730 pages in file cache
730 pages in page cache
42 pages in page table cache
Buffer memory: 5240kB
Buffer heads: 5278
Buffer blocks: 5240
Buffer hashed: 5238
CLEAN: 5240 buffers, 43 used (last=5239...
Networking buffers in use : 76737...
Total network buffer allocations : 32635...
Total failed network buffer allocs : 94
IP fragment buffer size : 0

(This was typed in from a videocam-snapshot of the screen, right edge cut
some numbers in the middle.)

On the last try I was running "free" on 1sec intervals, the last output
before it got OOM-killed was this:

total used free shared buffers cached
Mem: 257688 255052 2636 1612 2188 4652
-/+ buffers/cache: 248212 9476
Swap: 259232 3276 255956

I couldn't get the full meminfo output from this run but got one line which
seemed quite interesting:

Networking buffers in use : 774499

Tested 2.2.10-ac12 on the same machine, worked fine. 2.2.10 on a single
processor P2, worked ok.

Description of the server program:

Mainloop listens on an UDP socket for requests and forks off children to
serve them.

Child processes work like this:

s = socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
connect(s,remote_host_address);
f = open(file_path,O_RDONLY);
while (!eof(f))
{
read(f,data);
write(s,data);
}
close(s);
close(f);

All machines are connected to a relatively lightly loaded 100Mbit segment,
both Linux machines that I've tested this on have a 3Com 905B network adapter.

Kernel .config for the kernel that has trouble attached.

I can provide more information if necessary and run tests with different
kernels too.

--
Ilpo Ruotsalainen - <lonewolf@iki.fi> - http://www.iki.fi/lonewolf/

---559023410-851401618-934483831=:1744 Content-Type: APPLICATION/octet-stream; name="config-2.2.11-gunbuster.gz" Content-Transfer-Encoding: BASE64 Content-ID: <Pine.GSO.4.10.9908122150310.1744@hutcs> Content-Description: Content-Disposition: attachment; filename="config-2.2.11-gunbuster.gz"

H4sICIUTszcAA2NvbmZpZy0yLjIuMTEtZ3VuYnVzdGVyAI1ZW2/buBJ+768Q 0IezC+zi+BbbOUAfaIq2WVMiI8q3vghurKZGXCtry93m35+hZMe6DJUCBVLP fBwOh3Pj6OOHjw45p8mPTbp73Oz3r85TfIiPmzTeOl9fnR+b59j5ER/Oj8nh 2+7pf842OfwndeLtLv3w8QP8++jQZAuQTXo+7tJXZx//jPdO8pLuksMJ+FT6 Yz6J2EqxgHvMD4n49JovVMfkMT6dkqOTvr7Ezuawdb7FRk58ygXnS73usO/s Ts4hSZ1TnBYYPRvjroERalrkXTn9YR/0uvxaDfvRUkVLGcx0JGdlBvcXQk3K tJFeElUmKamIW1sMu5cJEyndiCheILcnI/jxpnRnMsIPQ8JpxLy5ICGXPnam MAhuUrWnroYXyWa7+bqHW0u2Z/hzOr+8JMf0dluedOeC6dtiICxYoGEfXVRt BvTixpn0SeY/e0M4v9xk+iy8yVOUF+XATzDEiEuNnjRnuzxgNETOmbOJvy7J j4y4MiWXUKY9zDlccpkmVcg9/oVVqMKNVCBpUXGPElTjBddLjaia+Y+s3Lch CkmJqND1Wi+4Ku030rkOTOuIUNQasIqG4iZlxP2xF0ZEzsMakYlxjeZxXd4y J38mC/yoigRKBiHKI8qruYfan5+yWFf7zWvB9W4CfVVbNNonj8/ONv65e4wL WWUkZpHLFtHYLRzjQuMuKx3jQp66hoNqW1jpcj1DbFtAUBeTDoyQqHfFj4VU av0eSlPNGzHUc/u9ViMk+NJutVrNR8mDETGCR7AzyvF4JEngIkLzRRGZhxJd CXHV77QbFQ4Dr3PffCZfDwe99l0jZsHJsENtdaBkQBwCR4nolCvNQiyQrxKE lAo7qj9yG3f2mtkB8Rr5q+blLqH3fezWIVjNwS4xW0rBGccqlaoHkNq46bRe Cvw4/Tc5Pu8OT/WOQBE6YyUVoEII7s/QTcaQupdECAtThCxAjjv3+erm2twv 78dVBNUz5JRoPH0BgLgL4lPmRgHkz/IeRVglZxVZDQtBH0BwfGnOnAT4nRjV BCd4vdRrP6JSzjjD+cYQ4GKBQkymZ6NIkGDCypZaWKJErfDED20efpGrDh64 gii8zRmBX05wIwhBUTqDvwy/0iXx37uRMbhDBrEipssIMvgSKADEXZKqecR1 pAFWi4qHRJsC+F9ofL9tdkfnn3N8jiFGikXQbKPplNVjKqsKtY7NUOtJXJeK FJ0GORU3zdsi3DRvqyeY18D2eTBFYu4XWhhDh//rkPhhqXXMOEJOJtyf4Gfc J//+nb8mtsfdz/hYehRkywdQ2uC2LOfJIIRWKlWFPSXtuw7uwzdAr9MMGPSw XJuzOR2sVoUkdCFEIX2IpB+N1mDUMQHL1SFQnXQEzyZjdk7Zp2Ehvi4Q0wvC fRK0Rl1BAYMiBkIEWX+6q6pv8hu0jPYDcr9T6SAqEry7Lr0f9OwIj01IQHjD NYzmECgTjkdzhnBD2u0MG9RgkHKauRG0Ju8gFJd2xHgezgMWudIj3LfDJm44 beAyH97BNPJpcNdtOg/3edikDfBJu+lesh1or9VvOLRem8vrtfHkXhAz+PXr Pchw1RBG+UaNEEV0kx7QpDb7oQF0Oi28Y84Rmnd6TYCHzAlNAXgXwzVetUty GvxZMzIhlhKTOzzt3rcaElvY7gzt3Hm7F3V74waACAOocxLP9bkGbDSvp+ZL S3d5i9ULERSuPF3p67SBBPQQp4XH2y2LBbRSpy8Md+5562K1YA9zaHe+NFRt Pao+dLKdWPo9Pprt/2i3HCi4APK+7tI/SwpHLJyyoDSgMNQF810ZRF0qvZIu An/BMIGXCSaUmOMeBaJwl2eii9K79K788LnQF9DNs1VRS2HaVqu1LkfTHu6i BUgAFRRvcYJQDNvde9yBuL63BOuaCWiKxpYUmnW9Gk9Z2U01MZU0Twr8ieC6 eODDE0/hHKUsIS4aOvaAuOWsnXlhmDzHB+do3kFIGIS4VwcUUhquwZSocihk 4pabg7M7pPHx26ayw1Tq0IyM9KKN+y6VFrNCnhJswUTUa1ke7q4oa5nH/I9N Gp+PTrDZ7hJsyDMlHm4qfoRn5un89fR6SuMf2FIelB+i+TLt+rdlJbjhVOFy v3Wo+3cgvWtz6fxhACbx/ZVB4TH8Z1EMdSNfYlOZHPR9c9w8gunrM6pFIass wqwZlqIwWtTQD2Qz8YLNDeWKtFxLBmGrEKKUWTqrHOPn3bdr64PNG/l+GKlw jfWQnpxrds3kHpyslL9DvhKmectQ+KOiielplH+NP33d/G3BsEMH7VZkl6ho G/iK4Ed94LTViawDuiUJ6dSVEzy4F7aZTBDSmj8sdts4ccZQcMTucP5VNNoC fEialr626HMCzrt7fMac/rNc65BT/E3thuXHdrZwnG5e4r8cqH/Ot33y8vLq GMK1budeX9xhXLVLLmW3j/OYKiWUh7m0tNtm9jfW0RgveMRt4I3tvGmNddWa wIO9+KFCu1LnlJuHX4lY4awK4Fre9/utEu2zFLzYHHjQfK8qm/hhg/bKpn72 fisLevBXPbsoyFg2WZAMOhVZ5guCTdLcvgmrsUqtn/EJR9edgkqX4Kr54+qd AAUbIOu5H6jCJxABFb0w29feqCqHKkxVtTmmOzNwdMLXl3ItNJ9QzHhfkBHD GxuPUDMODXn1u9rNrt57MiBzk4Dr/EtgoyyTf5fEPCwxgddjird22t+kELqO 2ByezpsnuIhaBy60uQqmyIRFve6gZK4ib9Ad4KmuBBpg/WYJMrxrWfcY3uFN cQWEzwQroN/QdtjHm84KCO9kKqDfUbyPt+kVED4bqYB+xwR9/LVQAeEteRlU ntcUIJD/hsO7+6hdvdIrw26WK8JukyvCbpArwm6NK8JuiivC7i9XBP6CLiLs tnwzlF3VmeTDqN6n0+RwSvZxYax5LUYTUu8P835BM5F/ur5lKZc0tojjUW1j nZwP28LMWM790oA4I0RMt7sDdKB5Y7cty9qmUTGflqKRlJbpRYbW0ud0wUeW LxQ5yNO+S4VtGvMGUbZxXIaQul4eZmYmsHe+bx6fK3N3j5jJDdTN4KG46v+o Wl9IJiMAAA== ---559023410-851401618-934483831=:1744--

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/