Re: Overscheduling DOES happen with high web server load.

Phillip Ezolt (ezolt@perf.zko.dec.com)
Fri, 7 May 1999 13:10:07 -0400 (EDT)


Greg,
For the record, I have been doing my performance tests on "strobe"
runs of SPECWeb96. This is a somewhat representative subset of the entire
suite. It runs much shorter than a full SPECWeb96 run. It allows one to
quickly figure what is happening in the system, and is ideal for performance
tuning.

I haven't yet tried a full run with Greg's changes, but on the strobe runs,
things do get better. Response time descreases and the maximum number of Ops
increases. (Both very good signs.)

I'll try a full run shortly.

--Phil

Digital/Compaq: HPSD/Benchmark Performance Engineering
Phillip.Ezolt@compaq.com ezolt@perf.zko.dec.com

On Fri, 7 May 1999, Greg Lindahl wrote:

> > Ok, you are right. The real problem is we are calculating goodness
> > O(A*B).
> >
> > A= Number of processes on the runqueue
> > B= Number of times schedule is called
>
> This is the amount of work we are doing, but I think you're on to the
> wrong solution. Phil did a test: he patched schedule() to pick the
> first schedulable process instead of the best one. That dropped the
> amount of time consumed in schedule() from 20% to 1%. That's the same
> as reducing the work from O(A*B) to O(B).
>
> Why is schedule called so frequently? The thundering herd. You get a
> new connection, everyone wakes up, only one gets work, and everyone
> else goes back to sleep, each causing O(A) work to reschedule. M hits
> per second, N processes, B=M*N. A=N. So the total work is O(M*N^2).
>
> These are separate problems. The thundering herd is fixed by
> wake_one. The cost of scheduling is still a problem; if we had
> many-cpu SMP linux boxes with high loads, all the cpus would sit
> around waiting for the scheduler lock. Of course, there would be other
> problems too. Since we can't fix all possible thundering herd
> situations, I think we should fix the scheduler too.
>
> Phil, did the SpecWeb score rise with the patch? It should have. Of
> course, that hacked scheduler is pretty broken...
>
> -- g
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/
>
>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/