Re: Benchmarks - 1.3.11

Randy Chapman (chapman@u.washington.edu)
Fri, 21 Jul 1995 14:04:52 -0700 (PDT)


On Thu, 20 Jul 1995, cjs wrote:

> > *******
> > Results
> > *******
> >
> > Pipe-based Context Switching Test || 3007.4 -> 4804.9 +59.77%
> > Pipe Throughput Test || 18574.4 -> 18774.0 +1.07%
> > Execl Throughput Test || 60.9 -> 61.4 +0.82%
> [stuff deleted]
>
> Do these numbers actually mean anything? You run one of these
> benchmarks every single time a kernel comes out, there is always
> minute differences in the numbers, but nothinging changes in the
> kernel except long --> _u32 or networking fixes or such. How does that
> explain all the differences in your numbers? Do you just run the test
> when you feel like it without reguard to what is swapped in or out? Do
> you neglect to flush the disk cache? Are you the only user online when
> you run them? 1.3.11 is an exception if you install that experimental
> patch linus gave, but for all the others I have to wonder why your
> tests show a variation when there is no kernel changes to support
> them.

Do you forget to read your mail, cjs? Perhaps you should go back and
read the message he posted explaining how the tests were made. Perhaps
you should also read the reports which always also include a test against
a 1.2.x kernel (1.2.11, i think..), which is always also interesting.

I think these reports are a great sanity check.. sure, there is a
quasi-normal -+10% variance... but it still serves as a sanity check for
everyone.

Please get a clue, a grip and a life and do something mroe useful ;)

Thanks!

--randy