Configuration:
plain 2.2.10 with and without the patch
all other config options the same
dual pentium II - 400
384MB RAM
SCSI disk with Adaptec controller, ASUS P2B
2GB ext2 file system with 1K blocks
When running with the patch I can repeatably lock up the computer
by running a utility which creates and writes a new file as fast
as it can.
fill junk 500000000 65536 66
create the file "junk"
write 500 million bytes to it
in write() calls of 64k at a time
using a buffer filled with char c = 66
I don't get a crash, it just hangs responding only to reset.
Writing 500MB always makes it hang. 100MB didn't.
300MB sometimes does. Competing with other things seems to
make the 300MB write more likely to hang. Plain 2.2.10
gets a bit unresponsive but never hangs.
I also did some testing using a program I wrote a year ago
to test whether I could use Linux for a project which requires
soft real time. The test runs two processes and a custom
shared memory queue library to pass messages between them.
The writer does usleep() and each time the timer expires it
writes a message with the current time down the queue. The
reader compares the time in the message with the time it
reads the message and calls the difference the delivery time.
The reader waits in select() when the queue is empty and is
awaken by a message on a Unix domain socket.
This is actually a very kind test since if the machine is
locked down doing some uninteruptable task the writer will
not get a chance to write a message in the first place.
A better test would use a genuinely independent start time.
All memory was locked down with mlockall().
Both processes in the test were run with real time priority.
No other processes in the system had real time priority.
Both tests start out competing with an "rm" of a 300MB file
which was recently written and is still mostly in the cash.
For the plain 2.2.10 test it competes with writing another
300MB file. For the patched test I kept getting lock ups
so I ran a kernel compile instead.
The following are histograms of delivery times.
Plain 2.2.10
Delivery Times Cumulative
125.893ms 0 0.00% 0 0.00%
112.202ms 1 0.02% 1 0.02%
100.000ms 0 0.00% 1 0.02%
89.125ms 0 0.00% 1 0.02%
79.433ms 0 0.00% 1 0.02%
70.795ms 0 0.00% 1 0.02%
63.096ms 0 0.00% 1 0.02%
56.234ms 0 0.00% 1 0.02%
50.119ms 0 0.00% 1 0.02%
44.668ms 0 0.00% 1 0.02%
39.811ms 0 0.00% 1 0.02%
35.481ms 0 0.00% 1 0.02%
31.623ms 1 0.02% 2 0.04%
28.184ms 0 0.00% 2 0.04%
25.119ms 0 0.00% 2 0.04%
22.387ms 0 0.00% 2 0.04%
19.953ms 0 0.00% 2 0.04%
17.783ms 0 0.00% 2 0.04%
15.849ms 1 0.02% 3 0.06%
14.125ms 0 0.00% 3 0.06%
12.589ms 0 0.00% 3 0.06%
11.220ms 0 0.00% 3 0.06%
10.000ms 1 0.02% 4 0.08%
8.913ms 0 0.00% 4 0.08%
7.943ms 0 0.00% 4 0.08%
7.079ms 0 0.00% 4 0.08%
6.310ms 3 0.06% 7 0.14%
5.623ms 4 0.08% 11 0.22%
5.012ms 48 0.96% 59 1.18%
4.467ms 31 0.62% 90 1.80%
3.981ms 13 0.26% 103 2.06%
3.548ms 12 0.24% 115 2.30%
3.162ms 6 0.12% 121 2.42%
2.818ms 3 0.06% 124 2.48%
2.512ms 5 0.10% 129 2.58%
2.239ms 0 0.00% 129 2.58%
1.995ms 0 0.00% 129 2.58%
1.778ms 5 0.10% 134 2.68%
1.585ms 1 0.02% 135 2.70%
1.413ms 2 0.04% 137 2.74%
1.259ms 1 0.02% 138 2.76%
1.122ms 1 0.02% 139 2.78%
1.000ms 1 0.02% 140 2.80%
0.891ms 1 0.02% 141 2.82%
0.794ms 0 0.00% 141 2.82%
0.708ms 1 0.02% 142 2.84%
0.631ms 1 0.02% 143 2.86%
0.562ms 1 0.02% 144 2.88%
0.501ms 0 0.00% 144 2.88%
0.447ms 1 0.02% 145 2.90%
0.398ms 5 0.10% 150 3.00%
0.355ms 2 0.04% 152 3.04%
0.316ms 3 0.06% 155 3.10%
0.282ms 9 0.18% 164 3.28%
0.251ms 10 0.20% 174 3.48%
0.224ms 4 0.08% 178 3.56%
0.200ms 10 0.20% 188 3.76%
0.178ms 10 0.20% 198 3.96%
0.158ms 14 0.28% 212 4.24%
0.141ms 20 0.40% 232 4.64%
0.126ms 9 0.18% 241 4.82%
0.112ms 19 0.38% 260 5.20%
0.100ms 37 0.74% 297 5.94%
0.089ms 66 1.32% 363 7.26%
0.079ms 64 1.28% 427 8.54%
0.071ms 283 5.66% 710 14.20%
0.063ms 615 12.30% 1325 26.50%
0.056ms 699 13.98% 2024 40.48%
0.050ms 2976 59.52% 5000 100.00%
0.045ms 0 0.00% 5000 100.00%
With Patch:
Delivery Times Cumulative
141.254ms 0 0.00% 0 0.00%
125.893ms 1 0.02% 1 0.02%
112.202ms 0 0.00% 1 0.02%
100.000ms 0 0.00% 1 0.02%
89.125ms 0 0.00% 1 0.02%
79.433ms 0 0.00% 1 0.02%
70.795ms 0 0.00% 1 0.02%
63.096ms 0 0.00% 1 0.02%
56.234ms 0 0.00% 1 0.02%
50.119ms 0 0.00% 1 0.02%
44.668ms 0 0.00% 1 0.02%
39.811ms 0 0.00% 1 0.02%
35.481ms 0 0.00% 1 0.02%
31.623ms 0 0.00% 1 0.02%
28.184ms 1 0.02% 2 0.04%
25.119ms 0 0.00% 2 0.04%
22.387ms 0 0.00% 2 0.04%
19.953ms 0 0.00% 2 0.04%
17.783ms 0 0.00% 2 0.04%
15.849ms 0 0.00% 2 0.04%
14.125ms 0 0.00% 2 0.04%
12.589ms 10 0.20% 12 0.24%
11.220ms 1 0.02% 13 0.26%
10.000ms 2 0.04% 15 0.30%
8.913ms 0 0.00% 15 0.30%
7.943ms 0 0.00% 15 0.30%
7.079ms 0 0.00% 15 0.30%
6.310ms 4 0.08% 19 0.38%
5.623ms 0 0.00% 19 0.38%
5.012ms 0 0.00% 19 0.38%
4.467ms 0 0.00% 19 0.38%
3.981ms 1 0.02% 20 0.40%
3.548ms 0 0.00% 20 0.40%
3.162ms 0 0.00% 20 0.40%
2.818ms 0 0.00% 20 0.40%
2.512ms 1 0.02% 21 0.42%
2.239ms 2 0.04% 23 0.46%
1.995ms 2 0.04% 25 0.50%
1.778ms 1 0.02% 26 0.52%
1.585ms 0 0.00% 26 0.52%
1.413ms 1 0.02% 27 0.54%
1.259ms 0 0.00% 27 0.54%
1.122ms 1 0.02% 28 0.56%
1.000ms 2 0.04% 30 0.60%
0.891ms 2 0.04% 32 0.64%
0.794ms 0 0.00% 32 0.64%
0.708ms 1 0.02% 33 0.66%
0.631ms 1 0.02% 34 0.68%
0.562ms 4 0.08% 38 0.76%
0.501ms 2 0.04% 40 0.80%
0.447ms 3 0.06% 43 0.86%
0.398ms 9 0.18% 52 1.04%
0.355ms 6 0.12% 58 1.16%
0.316ms 10 0.20% 68 1.36%
0.282ms 7 0.14% 75 1.50%
0.251ms 7 0.14% 82 1.64%
0.224ms 6 0.12% 88 1.76%
0.200ms 15 0.30% 103 2.06%
0.178ms 27 0.54% 130 2.60%
0.158ms 70 1.40% 200 4.00%
0.141ms 179 3.58% 379 7.58%
0.126ms 546 10.92% 925 18.50%
0.112ms 1047 20.94% 1972 39.44%
0.100ms 564 11.28% 2536 50.72%
0.089ms 131 2.62% 2667 53.34%
0.079ms 111 2.22% 2778 55.56%
0.071ms 279 5.58% 3057 61.14%
0.063ms 287 5.74% 3344 66.88%
0.056ms 1346 26.92% 4690 93.80%
0.050ms 310 6.20% 5000 100.00%
0.045ms 0 0.00% 5000 100.00%
I believe unlinking a large file is deadly to latency.
Both patched and unpatched I get >100ms delays as
a result. If I run similar tests competing against
CPU bound tasks or kernel compiles I get very few
events over 10ms.
The rest of the long latency events cut off very sharply
after 10ms. I guess it is related to the 10ms timer
triggered scheduling. Perhaps sometimes the availability
of data in the pipe does not immediately trigger run
readiness.
I am very glad someone is paying attention to latency in
Linux. My project is working well in Windows NT but when
I deal with installation and remote maintenance I often
wish for Linux.
If there is some way I can help please ask.
Mark Christiansen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/