Re: top hogs CPU in 2.6: kallsyms_lookup is very slow

From: Denis Vlasenko
Date: Fri Sep 17 2004 - 06:24:23 EST

> On Thu, Sep 16, 2004 at 02:57:08PM +0300, Denis Vlasenko wrote:
> > I recompiled 2.6 with HZ=100. It's not it.
> > Time is running normally too.
>
> Did the kallsyms patches reduce the cpu overhead from get_wchan()? I take
> this to mean reducing HZ to 100 did not alleviate the syscall problems?
> How do microbenchmarks fare, e.g. lmbench?

2x3 lmbench runs. HZ=100, configs attached.

i586-pc-linux-gnu: no data for Socket bandwidth using localhost
i586-pc-linux-gnu: no data for Socket bandwidth using localhost
i586-pc-linux-gnu: no data for Socket bandwidth using localhost
i586-pc-linux-gnu: no data for Socket bandwidth using localhost
i586-pc-linux-gnu: no data for Socket bandwidth using localhost
i586-pc-linux-gnu: no data for Socket bandwidth using localhost

L M B E N C H 3 . 0 S U M M A R Y
------------------------------------
(Alpha software, do not distribute)

Basic system parameters
------------------------------------------------------------------------------
Host OS Description Mhz tlb cache mem scal
pages line par load
bytes
--------- ------------- ----------------------- ---- ----- ----- ------ ----
hunter Linux 2.6.9-r i586-pc-linux-gnu 67 32 1
hunter Linux 2.6.9-r i586-pc-linux-gnu 67 32 1
hunter Linux 2.6.9-r i586-pc-linux-gnu 67 32 1
hunter Linux 2.4.27- i586-pc-linux-gnu 67 32 1
hunter Linux 2.4.27- i586-pc-linux-gnu 67 32 1
hunter Linux 2.4.27- i586-pc-linux-gnu 67 32 1

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host OS Mhz null null open slct sig sig fork exec sh
call I/O stat clos TCP inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
hunter Linux 2.6.9-r 67 2.27 5.25 106. 129. 12.7 64.3 4312 31.K 86.K
hunter Linux 2.6.9-r 67 2.27 4.77 100. 122. 13.3 62.2 4338 31.K 85.K
hunter Linux 2.6.9-r 67 2.27 5.48 107. 129. 14.3 62.5 4420 31.K 86.K
hunter Linux 2.4.27- 67 2.26 4.98 3073 3110 9.98 23.0 4200 77.K 240K
hunter Linux 2.4.27- 67 2.26 4.88 3034 3028 12.3 23.0 3852 78.K 237K
hunter Linux 2.4.27- 67 2.32 5.15 3009 3192 10.0 23.7 4047 78.K 238K

Context switching - times in microseconds - smaller is better
-------------------------------------------------------------------------
Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw
--------- ------------- ------ ------ ------ ------ ------ ------- -------
hunter Linux 2.6.9-r 31.9 64.0 120.8 115.7 322.2 136.0 356.4
hunter Linux 2.6.9-r 29.1 47.2 76.7 102.3 321.5 136.0 354.2
hunter Linux 2.6.9-r 29.3 56.0 144.5 101.9 305.5 145.3 351.0
hunter Linux 2.4.27- 19.8 39.4 190.3 77.8 368.0 104.1 401.5
hunter Linux 2.4.27- 19.7 30.9 105.0 87.2 316.9 107.2 359.9
hunter Linux 2.4.27- 19.5 34.6 95.5 74.3 279.1 109.5 325.0

*Local* Communication latencies in microseconds - smaller is better
---------------------------------------------------------------------
Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP
ctxsw UNIX UDP TCP conn
--------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
hunter Linux 2.6.9-r 31.9 129.8 230.
hunter Linux 2.6.9-r 29.1 130.1 244.
hunter Linux 2.6.9-r 29.3 136.9 233.
hunter Linux 2.4.27- 19.8 74.0 146.
hunter Linux 2.4.27- 19.7 74.4 134.
hunter Linux 2.4.27- 19.5 77.8 137.

File & VM system latencies in microseconds - smaller is better
-------------------------------------------------------------------------------
Host OS 0K File 10K File Mmap Prot Page 100fd
Create Delete Create Delete Latency Fault Fault selct
--------- ------------- ------ ------ ------ ------ ------- ----- ------- -----
hunter Linux 2.6.9-r 837.5 1272.3 2421.3 1751.3 1289.0 1.145 18.9 129.6
hunter Linux 2.6.9-r 862.8 1234.6 2463.1 1683.5 1270.0 3.732 18.4 128.1
hunter Linux 2.6.9-r 805.2 1122.3 2283.1 1550.4 1157.0 3.415 18.3 128.0
hunter Linux 2.4.27- 4219.4 4807.7 5586.6 4444.4 910.0 10.3 14.6 164.1
hunter Linux 2.4.27- 3759.4 4065.0 5586.6 4444.4 870.0 10.3 15.8 163.6
hunter Linux 2.4.27- 4048.6 4830.9 5714.3 4761.9 985.0 9.553 16.7 163.1

*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------------------------
Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem
UNIX reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
hunter Linux 2.6.9-r 13.1 11.1 16.8 54.3 18.4 18.5 54.2 26.1
hunter Linux 2.6.9-r 12.7 12.1 16.4 54.3 18.5 18.5 54.3 26.2
hunter Linux 2.6.9-r 13.0 11.2 18.3 54.3 18.5 18.5 54.3 26.1
hunter Linux 2.4.27- 15.7 11.9 17.6 54.2 18.6 18.6 54.4 26.0
hunter Linux 2.4.27- 15.7 10.7 18.3 54.2 18.4 18.5 54.4 26.1
hunter Linux 2.4.27- 15.5 10.9 17.8 54.3 18.6 18.5 54.4 26.1
--
vda

Attachment: c.tar.bz2
Description: application/tbz