At seeing the size of the latest patch (~270k), which I am currently,
happily running, I had the curiosity to find out how much (and how)
such vast changes affect kernel performances.
So, I decided to compile the Byte Unix benchmarks (from Sunsite) and
give them a go on the machine for different releases. I ran the tests
for 1.2.10, 1.3.3 and 1.3.4 and collected the relative log files.
This morning I decided that it would be nice to compare the different
log files in a somehow automatic way. I thought that our friendly kernel
developers could gain precious insights from the resulting rankings, and
thus give us a kernel that is everytime more outstanding and makes all
of us sooo proud :-P. Not having a lot of work to do today, I wrote a
small program that does exactly this.
The source of the program follows at the end of the message, in
gzipped, uuencoded format.
Here are the program outputs for the two transitions (1.2.10 -> 1.3.3
and 1.3.3 -> 1.3.4):
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
From:
* Linux pimpinel.fluido.org 1.2.10 #59 Tue Jun 13 09:46:17 MET DST 1995 i486
* Tue Jun 27 10:18:13 MET DST 1995
To:
* Linux pimpinel.fluido.org 1.3.3 #2 Mon Jun 19 21:16:39 MET DST 1995 i486
* Tue Jun 27 11:46:03 MET DST 1995
*******
Results
*******
Execl Throughput Test || 51.6 -> 60.8 +17.83%
Shell scripts (1 concurrent) || 78.3 -> 86.6 +10.60%
Shell scripts (8 concurrent) || 10.0 -> 11.0 +10.00%
Shell scripts (2 concurrent) || 41.0 -> 45.0 +9.76%
Shell scripts (4 concurrent) || 21.0 -> 23.0 +9.52%
File Copy (10 seconds) || 1180.0 -> 1287.0 +9.07%
File Copy (30 seconds) || 1054.0 -> 1140.0 +8.16%
Dhrystone 2 without register variables || 46284.2 -> 49771.4 +7.53%
C Compiler Test || 42.2 -> 45.2 +7.11%
File Write (30 seconds) || 8911.0 -> 9422.0 +5.73%
System Call Overhead Test || 29238.2 -> 30438.1 +4.10%
File Write (10 seconds) || 8100.0 -> 8333.0 +2.88%
File Read (30 seconds) || 93460.0 -> 95967.0 +2.68%
Process Creation Test || 119.9 -> 123.0 +2.59%
Dc: sqrt(2) to 99 decimal places || 5906.2 -> 5996.2 +1.52%
Recursion Test--Tower of Hanoi || 709.2 -> 719.4 +1.44%
Pipe Throughput Test || 18512.0 -> 18751.1 +1.29%
File Read (10 seconds) || 92743.0 -> 93080.0 +0.36%
Dhrystone 2 using register variables || 49802.4 -> 49954.4 +0.31%
Arithmetic Test (type = arithoh) || 126914.4 -> 126997.9 +0.07%
Arithmetic Test (type = long) || 8353.5 -> 8358.6 +0.06%
Arithmetic Test (type = int) || 8353.4 -> 8358.3 +0.06%
Arithmetic Test (type = register) || 8353.2 -> 8357.9 +0.06%
Arithmetic Test (type = short) || 7496.0 -> 7499.9 +0.05%
Arithmetic Test (type = double) || 5056.8 -> 5059.4 +0.05%
Arithmetic Test (type = float) || 5055.6 -> 5056.9 +0.03%
Pipe-based Context Switching Test || 3446.9 -> 3083.9 -10.53%
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
From:
* Linux pimpinel.fluido.org 1.3.3 #2 Mon Jun 19 21:16:39 MET DST 1995 i486
* Tue Jun 27 11:46:03 MET DST 1995
To:
* Linux pimpinel.fluido.org 1.3.4 #4 Mon Jun 26 22:25:32 MET DST 1995 i486
* Tue Jun 27 18:12:20 MET DST 1995
*******
Results
*******
Process Creation Test || 123.0 -> 136.3 +10.81%
Dc: sqrt(2) to 99 decimal places || 5996.2 -> 6466.2 +7.84%
Execl Throughput Test || 60.8 -> 64.1 +5.43%
Pipe-based Context Switching Test || 3083.9 -> 3184.1 +3.25%
System Call Overhead Test || 30438.1 -> 31302.2 +2.84%
File Write (10 seconds) || 8333.0 -> 8567.0 +2.81%
Shell scripts (2 concurrent) || 45.0 -> 46.0 +2.22%
File Write (30 seconds) || 9422.0 -> 9600.0 +1.89%
Shell scripts (1 concurrent) || 86.6 -> 87.9 +1.50%
Pipe Throughput Test || 18751.1 -> 19010.8 +1.38%
File Copy (10 seconds) || 1287.0 -> 1294.0 +0.54%
Recursion Test--Tower of Hanoi || 719.4 -> 721.1 +0.24%
Arithmetic Test (type = float) || 5056.9 -> 5058.2 +0.03%
Arithmetic Test (type = short) || 7499.9 -> 7500.8 +0.01%
Arithmetic Test (type = double) || 5059.4 -> 5059.8 +0.01%
Arithmetic Test (type = int) || 8358.3 -> 8358.3 +0.00%
Arithmetic Test (type = long) || 8358.6 -> 8358.6 +0.00%
Shell scripts (4 concurrent) || 23.0 -> 23.0 +0.00%
Shell scripts (8 concurrent) || 11.0 -> 11.0 +0.00%
Arithmetic Test (type = arithoh) || 126997.9 -> 126991.8 -0.00%
Arithmetic Test (type = register) || 8357.9 -> 8356.4 -0.02%
Dhrystone 2 using register variables || 49954.4 -> 49911.6 -0.09%
File Copy (30 seconds) || 1140.0 -> 1134.0 -0.53%
C Compiler Test || 45.2 -> 44.6 -1.33%
Dhrystone 2 without register variables || 49771.4 -> 48973.2 -1.60%
File Read (30 seconds) || 95967.0 -> 91729.0 -4.42%
File Read (10 seconds) || 93080.0 -> 86913.0 -6.63%
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
Please note that the *fluido.org* domain exists only in my mind - I
have a two-machine network that runs a domain name server :-) Very
useful when I connect to my server via PPP, in any case.
All the results are in loops-per-minute, loops-per-second or
KB-per-second. See the Byte benchmarks docfiles (somewhere on
Sunsite...) for details about the tests. Higher numbers always
indicate a better result. A fixed number of testing loops (3 or 6) is
performed for each loop, which runs for a fixed amount of time (10, 30
or 60 seconds).
The machine I run the tests on is an infamous, nameless clone with a
486DX2/66, 20MB of memory and two IDE disks - a Conner CP30204 and a
Maxtor 7546 AT (202 and 522 MB). It has no DOS partition ;-) and stays
on virtually forever.
The tests were started by rc at boot time, and I made sure that the
machine was (almost) not used for anything else for the length of the
tests (more than a hour each!!). For your enjoyment, here is the
difference w.r.t. the logfile that was left in the sunsite package by
its compiler, jon@robots.ox.ac.uk (I am sending a copy of this message
to his address too - hope he is still there). It refers to V0.95a, and
was run on a 386/33, 8MB of memory, more than 3 years ago:
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
From:
* Linux isabel 0 0.95a i386
* Sun Apr 5 21:33:45 1992
To:
* Linux pimpinel.fluido.org 1.3.4 #4 Mon Jun 26 22:25:32 MET DST 1995 i486
* Tue Jun 27 18:12:20 MET DST 1995
*******
Results
*******
File Write (30 seconds) || 133.0 -> 9600.0 +7118.04%
File Write (10 seconds) || 200.0 -> 8567.0 +4183.50%
File Copy (30 seconds) || 43.0 -> 1134.0 +2537.21%
File Copy (10 seconds) || 78.0 -> 1294.0 +1558.97%
Pipe Throughput Test || 1494.7 -> 19010.8 +1171.88%
File Read (10 seconds) || 31775.0 -> 86913.0 +173.53%
Arithmetic Test (type = double) || 1898.0 -> 5059.8 +166.59%
Dhrystone 2 using register variables || 19482.3 -> 49911.6 +156.19%
Dhrystone 2 without register variables || 19400.3 -> 48973.2 +152.44%
Arithmetic Test (type = float) || 2021.5 -> 5058.2 +150.22%
Shell scripts (8 concurrent) || 4.9 -> 11.0 +124.49%
Arithmetic Test (type = arithoh) || 60602.4 -> 126991.8 +109.55%
Shell scripts (4 concurrent) || 11.0 -> 23.0 +109.09%
Arithmetic Test (type = int) || 4032.9 -> 8358.3 +107.25%
Shell scripts (2 concurrent) || 22.2 -> 46.0 +107.21%
Arithmetic Test (type = long) || 4065.3 -> 8358.6 +105.61%
Shell scripts (1 concurrent) || 42.9 -> 87.9 +104.90%
Arithmetic Test (type = register) || 4133.4 -> 8356.4 +102.17%
File Read (30 seconds) || 45874.0 -> 91729.0 +99.96%
Arithmetic Test (type = short) || 3810.8 -> 7500.8 +96.83%
Recursion Test--Tower of Hanoi || 440.4 -> 721.1 +63.74%
System Call Overhead Test || 20149.6 -> 31302.2 +55.35%
Dc: sqrt(2) to 99 decimal places || 6267.5 -> 6466.2 +3.17%
Process Creation Test || 145.4 -> 136.3 -6.26%
Pipe-based Context Switching Test || 3928.4 -> 3184.1 -18.95%
Execl Throughput Test || 85.4 -> 64.1 -24.94%
C Compiler Test || 6109.4 -> 44.6 -99.27%
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
The result for the C Compiler test is quite strange...
Talking about compilers, I made my thansition from 2.5.9 to 2.6.3 (and
relative libraries) between compiling 1.3.3 and 1.3.4. Also, 1.3.1-2
are missing because I had to wait for the right time to download 2 1/2
MB of data... Why was the patch to 1.3.1 missing?!?
I am (still) running a.out. Had no time to try to understand enough of
ELF still. But will have to, I am afraid...
My intention is to run the benchmarks after I compile each new patch
(I did not remove the command from the rc file...) and post the
results to this mailing list. Will do so if I will be submerged by
tons of encouraging replies OR if I will NOT be sumberged by tons of
insults. So, if you hate this thing, light your flamethrower heheh.
The program follows after the signature. If you want to use it, just
compile it with
gcc -O2 -o bm_diff bm_diff.c
and run it as follows:
bm_diff first_logfile second_logfile > result
The benchmark run leaves a file called "log" in the
<bm-root-dir>/results directory. This is the file that you must save
and use for comparing.
Enjoy!
Carlo
-- * ...Ma appena fuori tutto e' gomma, * K * Carlo E. Prelz - fluido@telepac.pt tutto e' cicca impiastricciata... * (Marco Zappa-Niente cicca nella scuola)begin 644 bm_diff.c.gz M'XL("$1V\B\"`V)M7V1I9F8N8P"=%_U/VT;TY^2O>&1+L8-CDK!.A1#VT=&I M4K=)D$[;($*.?4Y.=>[<NS.,K?SO>^_NG,0$JFH(DKM[W]^/PQ[,5S<9S_,X MA=YANWW8:T,/1L>'@V\/A\?'+R&-60RE8L4_!'@MRWO%%TL#P>L0'B%`'_*B MXIG\WK""E4D:EX:(+EC&M5%\7IED7C!(TE2JC(L%&`EFR>#G7]_#@@FFD@+* M:E[P%/"/B91%\#M3\8BX.$Y)IJ&0"Y"5*2NC(5=R!>9.PH_WAL%[P?^&.1(N M5XGZ`*H2&A*1$:E*Q`=MI9&U3!%WNB<&[O`&N:PL(N&^U\F"G=2>(7DW0_LY M\@A?OY.+DXWGHEOXVJEWRS67`H;Q"*QWR(VC8X#!\<DW+T^^&0!\2%3A<`N6 M:.89;E$.=RB/AOB[H7PKN.'H*>5I'`N,W5=<I$65,3C5)N,R7IYM/54"8Y`U MW_"AX//';PHC0V_MKS*6<\'@EQ_^N)F>7TXOX6BP?KS\\W)Z_LO-Y?0".I?W MVK`5]/O06<-_^F%Z[J`_KN-Q48D31#'W)4,L0%%5:MK_M@$P4'PA6`;I,E'0 MRYA.1;)B8P3EA<08W2;%U6@6E:D9MQ_0\]F--NFXW>;"N)LR-\,@C.KS*`C' M[5O),^CI)$^*0J8!X2[EW2H1]S4P7<HR<#+GN<)7][Q*N+#HB5JDD8/C\?9J M%CZA+E)>C5Y^.XMZ66*8)CU[VKK$G4NC[,>(S'GS]MTY]')9TL7;0=]7:R_/ MK!&H<-2C`Q(2*FG#(WY-!E`M4%1_S@#0\/0KQ0HJANTJ,G\D#C"U3*NKX M7.YBJ7"E#?)=Y!R+4+-4BJR^7HM.9`T<S,*QY</^YB88VLL#B<JE"G(Q&8QS M<3K"CX.#C4Q4(T"+)OC'1&#Y(,)P%G54)PPGDU_?OWL76DR'#U"B;LAP@^JE M-N1ZR0!W2U0QR!?,Z`"=':&S(Y05[CW!V4;4Q](UL[<@17$/=PEZ4#%J/1KN MN%E"`I_0K9IGV,"F2W:/R5%DMD6@![$G:$,]BHM<VOIR_/T7F8R1F6`.ITME MM=K_M+\QMH7>1?**C=M-BA&1X"^=HTT1A;4Q<-BS*K@$`JH$Q4M#O:&@NEHK M\F^[1>P.B%^!;M_B-6ZWZOS+Q6RR+@"/263AP="BH?KE?;"%'5EH'8V'.BJ% M9D]:4)?YCOZ&KQC('./)TLJJ_ZSF:QZHD*N@+]-ZC?MYG1%X1@E!FB6`)*:A MBB^VN.XY&\%$V2?*M51!8A\31!3\&G6\R_#*PV:3_>O!/N9#"Y7:LY4,GSY! MX(M\,M<LP60*7M3EC]^NX"/-_V$R#WR[V'2YX5;"H2DVR2S%V63=3D)ZM\#= MYM#5)S"5$J@E6L_HK6$8XZ1)%0VIS02(FXVBU:C8%OF^Y;H:.8!Z]F`VV;X. M9Y-!/!A[K"NKZ\'!&L<"/FJI3/`EQM=\_K_[D`.J3)E"K'+%V$YX"6>KTO%X M,+2U_NP[)9K^P$L88<]E!;8;S+=6+]C!#%U&$#[V-ELVEL"U)[HF"D\K9G`C M6K'$UE!+ZS01N6?3Z>:=Z(7W0?^,?(P%\:@6W&>>%E(S:M2^J^_FPQN*.W0S MEPLVV-:#H>UB'KGS!G>NDVO1PZ%2?Q)FW40&L\A5II\E-=E4?H9H6!/945!W M;@$II9^6V$L>IR=-)-?9);9%V@EID$5.'#EO!4M"#O125NA3P6Z9@F52XHR* MXSBT`G(0TD20RA5ND\RZ'$<3KIX&IZ;;JMSHXSCY^*EUQI@_FGU[E,I\YO.= M:GK[!0VJAYYW1+=_]"H^>J5Q7W8.MU9P`5==/2.G>.IU@PD:`D+X#FP,.H#; MU%1VZHE-6>PD>7S<EB9!0Y5^DU-O.,!R/&P\^HGO(D"EN+NOXVBL"F-7:V=4 M[:DO+=V1+Y)GN"<9[7L5TYYM[;B>^[D6%TX!3*3Z!=UFW?!\N&RHFI%Z\0*^ M+%`8TNZK>)A#_\P?N@>OXE'>[3X5KH:0J"$A<BUS*T"4[52*CS;9>C/L)='Z M.'?+IV*F4H*VYW15!DG_;"-X<T:V#\WM>/0YGF[)IO]F)L@$U>HG]JO>,@ER M-G#^\?*'XPWHM`GJ6YB_#*PFGUG%K0(.;G==QW9O#7YNLSV_N/CM8F]O#_YB M2O9Q15C03D?L4?3'N,Z(W56VWMT")S1<.9UJ@5L+JQ/LE]5.K?V3?-?V6AMJ FBQ_]?V%-=5>_UK<;J^1FE[P6^V%COR4"OT`\M/\#,'88?;</``"3 ` end