Re: Benchmarks - 1.3.11

Carlo Emilio Prelz (fluido@telepac.pt)
Thu, 20 Jul 1995 23:39:00 +0200 (MET DST)


On Thu, 20 Jul 1995, cjs wrote:

> > *******
> > Results
> > *******
> >
> > Pipe-based Context Switching Test || 3007.4 -> 4804.9 +59.77%
> > Pipe Throughput Test || 18574.4 -> 18774.0 +1.07%
> > Execl Throughput Test || 60.9 -> 61.4 +0.82%
> [stuff deleted]
>
> Do these numbers actually mean anything? You run one of these
> benchmarks every single time a kernel comes out, there is always
> minute differences in the numbers, but nothinging changes in the
> kernel except long --> _u32 or networking fixes or such. How does that
> explain all the differences in your numbers? Do you just run the test
> when you feel like it without reguard to what is swapped in or out? Do
> you neglect to flush the disk cache? Are you the only user online when
> you run them? 1.3.11 is an exception if you install that experimental
> patch linus gave, but for all the others I have to wonder why your
> tests show a variation when there is no kernel changes to support
> them.

Duh... What I do is, *always*, start the benchmark run (4 to 5 hours)
after a fresh reboot. I login from the first multiterminal, start the
daemon, check that it started OK, log out, and, generally, go to sleep.

My machine is not on the net (except when I dial in). From MY point of
view, I am being as impartial as possible.

Each test is run 20 times (10 for the most time-consuming). If I find the
time, I will add the calculation of the standard deviation.

Ciao
Carlo

--
  *                                          ...Ma appena fuori tutto e' gomma,
* K * Carlo E. Prelz - fluido@telepac.pt      tutto e' cicca impiastricciata...
  *                                     (Marco Zappa-Niente cicca nella scuola)