Re: 2.6.0-test6 scheduler goodness

[Nick Piggin]

David B Harris wrote:

> Hey guys. Using Linux 2.6.0-test6, and have a few scheduler comments
> (first though, good work! :) see below):
>
> 1) I'm a fairly heavy user/developer. 2.4.x's stock scheduler has always
> been excellent for me. Just in case I was missing out, I tried preempt
> patches, lowlatency patches, different VMs/schedulers and such. All of
> them reduced throughput and responsiveness noticeably.
>
> 2) I've been tracking 2.5.x/2.6.0-test* for a while, and up to and
> including 2.6.0-test4, I had the same problems as I did with non-stock
> 2.4.x schedulers; in other words, both throughput and responsiveness
> were worse than I needed (in other words, they weren't as good as what I
> was used to).
>
> 3) I'm aware of how easy it is to create a world-class workstation
> scheduler, and how hard it is to make a scheduler that's excellent in
> all workload cases ;)
>
> 4) In either 2.6.0-test5, or 2.6.0-test6 (I'm using 2.6.0-test6, I
> skipped test5), responsiveness was magically fixed for my workload case.
> I still have lower throughput, apparently (big compiles and whatnot take
> about 20% longer), but I recently got a CPU upgrade so I don't
> particularily notice it - the CPU upgrade was a pretty major one.
>
> Allright, so, in that context, here are some excerpts form a
> conversation I had on IRC:
>
> [14:42:14] <ElectricElf> Heh, impressive:                                     
> [14:42:14] <ElectricElf>  14:41:55 up 14:58,  6 users,  load average: 171.82, 319.28, 475.21
> [14:44:51] <ElectricElf> Oh, heh, that explains why:
> [14:44:59] <ElectricElf> [ david@willow: ~/ ]$ ps auwwx | wc -l
> [14:44:59] <ElectricElf> 3381
> [14:45:22] <steve> mothers of christ
> [14:45:31] <ElectricElf> They're all procmail instances.
> [14:45:34] <ElectricElf> formail/procmail sucks.
> [14:45:40] <ElectricElf> Highest loadavg I've ever seen though.
> [14:45:47] <steve> same.
> [14:46:40] <ElectricElf> In 2.6.x's defence, though, I didn't even notice it
>                         was that bad.
> [14:46:51] <ElectricElf> Still web browsing fairly normally.
> [14:47:04] <ElectricElf> Disk was thrashing, but I just assumed it was
>                         slugging through all the mails sequentially.
> [14:47:50] <ElectricElf> I think I have about 3200 procmail processes polling
>                         for a lockfile every second.
> [14:48:02] <ElectricElf> ... and one of 'em died, not releasing lockfile :)
> [14:48:33] <steve> hahaha
> [14:48:46] <ElectricElf> [ david@willow: ~/ ]$ ps auwwx | wc -l
> [14:48:46] <ElectricElf> 3379
> [14:48:51] <ElectricElf> Yah. They're all just kind of sitting there.
> [14:48:59] *  ElectricElf removes the lock and waits for the shitstorm
> [14:49:15] <ElectricElf> Heh. Didn't do much at all.
> [14:49:18] <ElectricElf> *sigh*
> [14:49:24] *  ElectricElf killalls procmail
> # Annotation:
> # I had suspended the formail process which had all the procmails traced.
> #
> # A few processes were very, very unhappy at this point: notably, my Screens 
> # and X were unable to process keyboard/mouse input. There was quite the 
> # swapstorm as procmail processes which had been swapped out were brought back
> # in and killed. However, it passed in about 30 seconds, and I was able to
> # watch the progress as my pretty much everything appeared to at least be able
> # to continue with output; gkrellm and X didn't appear to be struggling in 
> # that manner.

Hi David,
You said your box was swapping heavily - this is probably the cause of your
problems. Can you reproduce bad behaviour when the box is not swapping?

Try my scheduler patches if you like.
http://www.kerneltrap.org/~npiggin/v15a/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/