Summary:
Caching in 2.3.99-pre* is so aggressive, it induces a vast
amount of paging, crippling performance on low-RAM machines or
under high memory load.
In followup to a previous post on this topic "Zealous paging...",
here's an example that illustrates this behavior. On a 32 MB i586 system
running X with a single xterm open, I have 'vmstat' dumping statistics
every 7 seconds as Netscape, gimp, GNU emacs, and a second instance of
gimp are sequentially loaded. Then all apps are sequentially closed. Very
simple.
Here's the results from vmstat for 2.3.99-pre2-3 during this process:
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 0 3368 1116 16340 0 0 34 2 118 76 9 3 88
1 0 1 0 872 556 17240 0 0 697 0 128 91 46 27 27
1 0 0 0 804 512 17012 0 0 274 9 120 146 63 14 23
1 0 0 88 776 424 16320 0 18 401 6 134 548 50 26 25
2 0 0 344 848 400 15580 0 51 432 15 154 331 41 27 32
2 0 0 2744 876 404 15900 0 305 377 81 199 249 18 29 54
1 0 0 3128 800 440 16072 0 77 124 23 139 247 58 17 25
2 0 0 4552 784 400 16524 0 94 39 25 115 677 52 38 10
1 0 0 5312 776 408 16436 0 152 149 42 133 154 61 19 20
2 0 0 6624 776 420 16696 22 184 1013 46 169 109 2 25 74
1 0 0 7480 860 424 17048 0 171 751 45 177 119 7 22 71
1 0 0 8116 876 436 16916 59 94 198 24 145 354 51 19 30
2 2 0 8660 396 396 17080 322 111 349 35 174 514 26 27 47
2 0 0 10116 852 412 18064 258 107 419 30 195 239 7 20 73
1 1 0 10380 848 460 17148 117 60 292 16 160 178 13 20 68
3 0 0 10600 788 400 16160 42 53 101 16 120 211 79 15 6
2 0 0 11032 776 404 16064 26 89 293 27 182 331 48 24 28
2 0 1 12024 812 404 16088 146 131 564 36 166 119 30 30 40
4 0 0 12340 788 400 15812 66 71 406 20 150 231 42 28 30
2 0 0 12980 796 400 16100 98 78 322 22 1665 1326 1 1 97
1 1 0 13248 784 400 16400 247 87 476 24 212 270 4 18 78
1 0 0 11672 880 416 21820 266 0 622 12 179 208 6 19 75
0 0 0 8340 1036 492 22300 226 0 459 2 156 129 6 16 78
And here's the result of the same operations in kernel 2.2.13:
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 0 7820 1080 15328 0 0 147 7 124 127 33 15 52
1 0 1 5152 848 1136 25076 0 1038 744 271 157 100 30 9 62
2 0 0 3364 680 1100 22316 26 48 726 13 142 202 34 13 53
2 0 0 3220 336 1044 21576 98 0 328 3 128 646 64 12 24
2 0 0 3824 572 660 19228 3 192 446 58 200 531 36 19 45
1 0 0 3884 548 664 18772 0 12 108 3 134 238 75 11 14
2 0 0 3972 636 676 16216 2 0 180 5 137 233 76 12 12
2 0 0 4536 524 652 15804 22 115 246 34 130 137 83 9 8
1 1 0 4580 844 652 13416 45 10 476 5 221 738 44 21 35
2 0 0 4676 508 652 14012 46 26 909 13 196 289 18 12 70
3 0 0 4840 780 656 11820 10 16 120 6 127 538 86 13 1
2 0 0 4900 548 652 11324 18 18 71 9 121 586 75 16 9
2 0 0 6184 788 656 11688 184 247 349 71 878 497 1 1 98
2 0 0 6172 1804 664 12488 50 0 126 0 180 349 10 5 86
0 0 0 4604 8172 780 17060 14 27 169 11 125 179 27 9 64
'free' w/ all apps loaded with 2.3.99-pre2-3:
total used free shared buffers cached
Mem: 30364 29592 772 0 360 17504
-/+ buffers/cache: 11728 18636
Swap: 32764 15016 17748
'free' w/ all apps loaded with 2.2.13:
total used free shared buffers cached
Mem: 31032 30164 868 16880 652 11976
-/+ buffers/cache: 17536 13496
Swap: 32764 6744 26020
Summary notes:
1) 2.2.13 takes 115 seconds to accomplish what 2.3.99 took 165
seconds to do. This is due to all the context switches
and paging...
2) 2.3.99 doesn't reduce cache size to account for the increased
memory load at all; swapping was necessary to make up the
difference.
3) Previous versions of 2.3 didn't show this behavior.
4) This affect is noted on larger RAM systems as well, but masked
by having more resources available. Loading
applications on a 128 MB system with 2.3.99-pre1 reduces cache
size, but only down to about 40 MB, at which point swapping
commences in earnest. This is well in advance of where previous
kernel releases would begin paging seriously.
This "feature" seems to cripple low-RAM systems. I have looked at the
patch from 2.3.51 to 2.3.99-pre1, especially the changes in mm/ and
didn't see an obvious culprit.
Can anyone shed light upon this new behavior? I'm genuinely interested
in how this works...
And consider me a _very_ willing tester of any patches that might help! :)
Craig Kulesa
ckulesa@as.arizona.edu
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Thu Mar 23 2000 - 21:00:24 EST