High load under low io with 2.4.19+

From: Matthew \"Cheetah\" Gabeler-Lee
Date: Mon Aug 25 2003 - 22:32:47 EST


The quick version: Almost any i/o goes slow and can trigger high load

I'm experiencing abnormally high load averages and matching system
slow-downs when doing even minimal i/o on my system. When this load
issue is bad, the system load can move towards 5 or 10, with little or
no cpu usage, and not much disk access (<1MB/s, no thrashing sounds).
When the load average gets high because of this kind of thing, the
system can become quite unresponsive.

Some empirical data points:

Compiling the kernel with -j2 (2-cpu system) while doing nothing else
intensive (various idle processes common to a desktop workstation) can
drive the load up towards 10. During this time, the two cpus in
combination use, on average, 50% of capacity. At a more detailed level,
what I often see is that the two cpus will hit 100% utilization for a
couple seconds, and then drop down to near zero for a couple seconds.
Every other multi-cpu system I've used, make -j ncpus on the kernel can
come very close to fully loading all cpus.

Running find / >/dev/null can bring the system almost to its knees, to
the point where the mouse cursor gets jittery in X. Running find /
without piping it to /dev/null shows frequent pauses in the output for
2-5 seconds at a time.

Starting up pine (I have some startup key sequences pine auto-runs
which do a couple things) can take 10+ seconds. On my previous system,
which was nominally much slower, these startup actions would usually
take < 1 sec.

Generally, any i/o that can't be satisfied from cache goes rather slow,
and any large amount of i/o drives the system load way up and makes the
system unresponsive (can take seconds to respond to mouse or keyboard
event).

According to sysstat/sar, my 5-minute load average hasn't dropped below
0.1 since this begun, even during the day when the system should be
completely idle. sar is reporting daily load averages almost never
going below 0.5. For the vast majority of the day (when I'm not
actively using it), this machine does almost nothing besides fetching
small amounts of e-mail and logging a couple small IRC channels.

Debian's 2.4.18-bf24 kernel that came with 3.0r1 doesn't behave
perfectly for me, but does behave quite a bit better. Self-built 2.4.18
through 2.4.22 all exhibit this issue. I've tried the -aa and -rmap
patches on .21, and they made no difference whatsoever. I've tried
using the noapic boot option, and that didn't help. I've tried
disabling HIGHMEM and HIGHIO, and that didn't help.

I verified that the ide-raid adaptor wasn't just going totally slow by
doing things like dd if=/dev/zero of=test, and was seeing ~30MB/sec
sustained (according to stats in /proc/partitions).

I've run similarly configured kernels on other systems with similar OS
configurations, but lower end hardware configurations (single cpu,
single ide hd, 512MB ram) and not seen these kinds of issues

Basic hardware information:
Dual Athlon MP 2400+
MSI K7D-Master L motherboard (AMD MPX Chipset)
1GB ECC RAM (Check+Scrub enabled in BIOS)
3Ware 7500-4 LP ATA-RAID Controller (3x 120GB drive, RAID5)

The BIOS is set to use MP spec 1.1, since I saw some things on the web
that indicated that 1.4 wouldn't work.

Some notes related to some kernel threads I came upon trying to solve
this bug:

MTRRs seem to be enabled OK: # cat /proc/mtrr
reg00: base=0x00000000 ( 0MB), size=1024MB: write-back, count=1
reg01: base=0xe8000000 (3712MB), size= 128MB: write-combining, count=1
reg05: base=0xe0000000 (3584MB), size= 128MB: write-combining, count=1

However, I'm seeing this in my dmesg every time I boot:
# dmesg | grep mtrr
mtrr: v1.40 (20010327) Richard Gooch (rgooch@xxxxxxxxxxxxx)
mtrr: detected mtrr type: Intel
mtrr: your CPUs had inconsistent fixed MTRR settings
mtrr: probably your BIOS does not setup all CPUs

3d accelleration on the nvidia card works quite well, however, so I
imagine mtrr has to be working more or less.

The symptoms (i/o pausing for several seconds) seem similar to those
described in the "get_request starvation bug" thread
(http://lkml.org/lkml/2002/2/8/73) and "design locking bug in
wait_on_page/wait_on_buffer/get_request_wait" thread
(http://lkml.org/lkml/2002/11/11/251) but those patches seems to be
integrated already in one form or another by 2.4.22. I don't see the
i/o pausing for hours at a time, but then again I'm not running abusive
benchmark programs trying to trigger such behavior.

Detailed hardware info as suggested by
http://kernel.org/pub/linux/docs/lkml/reporting-bugs.html attached as
hardware_info.txt

PS: please CC me on replies

--
-Cheetah
"Reality is that which, when you stop believing in it, doesn't go away".
-- Philip K. Dick
GPG pubkey fingerprint: A57F B354 FD30 A502 795B 9637 3EF1 3F22 A85E 2AD1# df -Tk
Filesystem Type 1k-blocks Used Available Use% Mounted on
/dev/sda1 ext3 482217 71991 385327 16% /
/dev/raid5/var
ext3 507748 338843 142691 71% /var
/dev/raid5/home
reiserfs 2097084 675096 1421988 33% /home
/dev/raid5/usr
reiserfs 8388348 6135520 2252828 74% /usr
/dev/raid5/grande
reiserfs 228074696 210020060 18054636 93% /mnt/grande

# ver_linux
Gnu C 2.95.4
Gnu make 3.79.1
util-linux 2.11n
mount 2.11n
modutils 2.4.21
e2fsprogs 1.27
PPP 2.4.1
Linux C Library 2.3.1
Dynamic linker (ldd) 2.3.1
Procps 2.0.7
Net-tools 1.60
Console-tools 0.2.3
Sh-utils 2.0.11
Modules Loaded nvidia agpgart lirc_i2c lirc_dev ip_conntrack_ftp
ipt_TOS ipt_limit ip_nat_irc ip_conntrack_irc ipt_state ipt_LOG
ipt_REJECT iptable_nat ip_conntrack iptable_mangle iptable_filter
ip_tables amd7xx_tco w83781d i2c-isa usb-storage visor usbserial emu10k1
sound ac97_codec tulip crc32 eepro100 mii tvmixer tuner tvaudio msp3400
bttv i2c-algo-bit videodev lvm-mod i2c-dev i2c-proc i2c-core keybdev
input usb-ohci

(emu10k1 and ac97_codec are from creative's CVS server)

# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 8
model name : AMD Athlon(tm) MP 2400+
stepping : 1
cpu MHz : 2000.090
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow
bogomips : 3984.58

processor : 1
vendor_id : AuthenticAMD
cpu family : 6
model : 8
model name : AMD Athlon(tm) MP
stepping : 1
cpu MHz : 2000.090
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow
bogomips : 3997.69

# cat /proc/modules
nvidia 1629280 11 (autoclean)
agpgart 18176 3 (autoclean)
lirc_i2c 2592 1
lirc_dev 7888 1 [lirc_i2c]
ip_conntrack_ftp 3888 0 (unused)
ipt_TOS 1080 22 (autoclean)
ipt_limit 1048 1 (autoclean)
ip_nat_irc 2320 0 (unused)
ip_conntrack_irc 3024 2 [ip_nat_irc]
ipt_state 600 3 (autoclean)
ipt_LOG 3288 6 (autoclean)
ipt_REJECT 3256 8 (autoclean)
iptable_nat 16792 3 (autoclean) [ip_nat_irc]
ip_conntrack 19944 7 (autoclean) [ip_conntrack_ftp ip_nat_irc ip_conntrack_irc ipt_state iptable_nat]
iptable_mangle 2196 1 (autoclean)
iptable_filter 1704 1 (autoclean)
ip_tables 11928 10 [ipt_TOS ipt_limit ipt_state ipt_LOG ipt_REJECT iptable_nat iptable_mangle iptable_filter]
amd7xx_tco 3036 1
w83781d 18748 0
i2c-isa 1164 0 (unused)
usb-storage 34552 0 (unused)
visor 10668 0
usbserial 16444 0 [visor]
emu10k1 70344 1
sound 56876 0 [emu10k1]
ac97_codec 10952 0 [emu10k1]
tulip 38464 1
crc32 2864 0 [tulip]
eepro100 18900 1
mii 2464 0 [eepro100]
tvmixer 3312 0
tuner 9472 1 (autoclean)
tvaudio 12732 0 (autoclean) (unused)
msp3400 15788 1 (autoclean)
bttv 97152 2
i2c-algo-bit 7112 1 [bttv]
videodev 5824 2 [bttv]
lvm-mod 62080 9
i2c-dev 3876 0 (unused)
i2c-proc 6288 0 [w83781d]
i2c-core 13156 0 [lirc_i2c w83781d i2c-isa tvmixer tuner tvaudio msp3400 bttv i2c-algo-bit i2c-dev i2c-proc]
keybdev 1728 0 (unused)
input 3616 0 [keybdev]
usb-ohci 17000 0 (unused)

(emu10k1 and ac97_codec are from creative's CVS server)

# cat /proc/ioports
0000-001f : dma1
0020-003f : pic1
0040-005f : timer
0060-006f : keyboard
0070-007f : rtc
0080-008f : dma page reg
00a0-00bf : pic2
00c0-00df : dma2
00f0-00ff : fpu
0290-0297 : w83627hf
02f8-02ff : serial(set)
0378-037a : parport0
037b-037f : parport0
03c0-03df : vga+
03f8-03ff : serial(set)
0cf8-0cff : PCI conf1
c000-cfff : PCI Bus #02
c000-c0ff : Linksys Network Everywhere Fast Ethernet 10/100 model NC100
c000-c0ff : tulip
c400-c41f : Creative Labs SB Live! EMU10k1
c400-c41f : EMU10K1
c800-c807 : Creative Labs SB Live! MIDI/Game Port
cc00-cc3f : Intel Corp. 82557/8/9 [Ethernet Pro 100]
cc00-cc3f : eepro100
d000-d00f : Advanced Micro Devices [AMD] AMD-768 [Opus] IDE
d400-d4ff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter
d400-d4ff : sym53c8xx
d800-d8ff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter (#2)
d800-d8ff : sym53c8xx
dc00-dc0f : 3ware Inc 3ware 7000-series ATA-RAID
dc00-dc0f : 3ware Storage Controller
e000-e003 : Advanced Micro Devices [AMD] AMD-760 MP [IGD4-2P] System Controller

# cat /proc/iomem
00000000-0009fbff : System RAM
0009fc00-0009ffff : reserved
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000d0000-000d3fff : Extension ROM
000d4000-000d4fff : Extension ROM
000d5000-000d67ff : Extension ROM
000f0000-000fffff : System ROM
00100000-3fffffff : System RAM
00100000-0028969d : Kernel code
0028969e-0032a887 : Kernel data
e0000000-e7ffffff : Advanced Micro Devices [AMD] AMD-760 MP [IGD4-2P] System Controller
e8000000-efffffff : PCI Bus #01
e8000000-efffffff : nVidia Corporation NV28 [GeForce4 Ti 4200 AGP 8x]
f0000000-f1ffffff : PCI Bus #01
f0000000-f0ffffff : nVidia Corporation NV28 [GeForce4 Ti 4200 AGP 8x]
f3000000-f4ffffff : PCI Bus #02
f4000000-f401ffff : Intel Corp. 82557/8/9 [Ethernet Pro 100]
f4020000-f4020fff : Advanced Micro Devices [AMD] AMD-768 [Opus] USB
f4020000-f4020fff : usb-ohci
f4021000-f4021fff : Intel Corp. 82557/8/9 [Ethernet Pro 100]
f4021000-f4021fff : eepro100
f4022000-f40223ff : Linksys Network Everywhere Fast Ethernet 10/100 model NC100
f4022000-f40223ff : tulip
f5000000-f57fffff : 3ware Inc 3ware 7000-series ATA-RAID
f5800000-f58fffff : PCI Bus #02
f5800000-f5800fff : Brooktree Corporation Bt878 Video Capture
f5800000-f5800fff : bttv
f5801000-f5801fff : Brooktree Corporation Bt878 Audio Capture
f5900000-f5901fff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter (#2)
f5902000-f5903fff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter
f5904000-f590400f : 3ware Inc 3ware 7000-series ATA-RAID
f5905000-f59053ff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter
f5906000-f59063ff : LSI Logic / Symbios Logic 53c1010 Ultra3 SCSI Adapter (#2)
f5907000-f5907fff : Advanced Micro Devices [AMD] AMD-760 MP [IGD4-2P] System Controller
fec00000-ffffffff : reserved

# lspci -vv
00:00.0 Host bridge: Advanced Micro Devices [AMD]: Unknown device 700c (rev 11)
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
Latency: 32
Region 0: Memory at e0000000 (32-bit, prefetchable) [size=128M]
Region 1: Memory at f5907000 (32-bit, prefetchable) [size=4K]
Region 2: I/O ports at e000 [disabled] [size=4]
Capabilities: [a0] AGP version 2.0
Status: RQ=15 SBA+ 64bit- FW+ Rate=x1,x2
Command: RQ=0 SBA- AGP+ 64bit- FW- Rate=<none>

00:01.0 PCI bridge: Advanced Micro Devices [AMD]: Unknown device 700d (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32
Bus: primary=00, secondary=01, subordinate=01, sec-latency=32
Memory behind bridge: f0000000-f1ffffff
Prefetchable memory behind bridge: e8000000-efffffff
BridgeCtl: Parity- SERR+ NoISA+ VGA+ MAbort- >Reset- FastB2B-

00:07.0 ISA bridge: Advanced Micro Devices [AMD]: Unknown device 7440 (rev 05)
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 0

00:07.1 IDE interface: Advanced Micro Devices [AMD]: Unknown device 7441 (rev 04) (prog-if 8a [Master SecP PriP])
Subsystem: Advanced Micro Devices [AMD]: Unknown device 7441
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32
Region 4: I/O ports at d000 [size=16]

00:07.3 Bridge: Advanced Micro Devices [AMD]: Unknown device 7443 (rev 03)
Subsystem: Advanced Micro Devices [AMD]: Unknown device 7443
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-

00:08.0 SCSI storage controller: LSI Logic / Symbios Logic (formerly NCR) 53c1010 Ultra3 SCSI Adapter (rev 01)
Subsystem: LSI Logic / Symbios Logic (formerly NCR): Unknown device 1040
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 72 (4250ns min, 4500ns max), cache line size 08
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at d400 [size=256]
Region 1: Memory at f5905000 (64-bit, non-prefetchable) [size=1K]
Region 3: Memory at f5902000 (64-bit, non-prefetchable) [size=8K]
Expansion ROM at <unassigned> [disabled] [size=128K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

00:08.1 SCSI storage controller: LSI Logic / Symbios Logic (formerly NCR) 53c1010 Ultra3 SCSI Adapter (rev 01)
Subsystem: LSI Logic / Symbios Logic (formerly NCR): Unknown device 1040
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 72 (4250ns min, 4500ns max), cache line size 08
Interrupt: pin B routed to IRQ 10
Region 0: I/O ports at d800 [size=256]
Region 1: Memory at f5906000 (64-bit, non-prefetchable) [size=1K]
Region 3: Memory at f5900000 (64-bit, non-prefetchable) [size=8K]
Expansion ROM at <unassigned> [disabled] [size=128K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

00:09.0 RAID bus controller: 3ware Inc 3ware 7000-series ATA-RAID (rev 01)
Subsystem: 3ware Inc 3ware 7000-series ATA-RAID
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (2250ns min), cache line size 08
Interrupt: pin A routed to IRQ 10
Region 0: I/O ports at dc00 [size=16]
Region 1: Memory at f5904000 (32-bit, non-prefetchable) [size=16]
Region 2: Memory at f5000000 (32-bit, non-prefetchable) [size=8M]
Expansion ROM at <unassigned> [disabled] [size=64K]
Capabilities: [40] Power Management version 1
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

00:10.0 PCI bridge: Advanced Micro Devices [AMD]: Unknown device 7448 (rev 05) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
Latency: 32
Bus: primary=00, secondary=02, subordinate=02, sec-latency=32
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: f3000000-f4ffffff
Prefetchable memory behind bridge: f5800000-f58fffff
BridgeCtl: Parity- SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-

01:05.0 VGA compatible controller: nVidia Corporation: Unknown device 0281 (rev a1) (prog-if 00 [VGA])
Subsystem: CardExpert Technology: Unknown device 0404
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 248 (1250ns min, 250ns max)
Interrupt: pin A routed to IRQ 10
Region 0: Memory at f0000000 (32-bit, non-prefetchable) [size=16M]
Region 1: Memory at e8000000 (32-bit, prefetchable) [size=128M]
Expansion ROM at <unassigned> [disabled] [size=128K]
Capabilities: [60] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [44] AGP version 3.0
Status: RQ=31 SBA+ 64bit- FW+ Rate=x1,x2
Command: RQ=15 SBA- AGP+ 64bit- FW- Rate=<none>

02:00.0 USB Controller: Advanced Micro Devices [AMD]: Unknown device 7449 (rev 07) (prog-if 10 [OHCI])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR+
Latency: 32 (20000ns max), cache line size 08
Interrupt: pin D routed to IRQ 5
Region 0: Memory at f4020000 (32-bit, non-prefetchable) [size=4K]

02:04.0 Ethernet controller: Linksys Network Everywhere Fast Ethernet 10/100 model NC100 (rev 11)
Subsystem: Linksys: Unknown device 0570
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (63750ns min, 63750ns max), cache line size 08
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at c000 [size=256]
Region 1: Memory at f4022000 (32-bit, non-prefetchable) [size=1K]
Expansion ROM at <unassigned> [disabled] [size=128K]
Capabilities: [c0] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

02:05.0 Multimedia audio controller: Creative Labs SB Live! EMU10k1 (rev 08)
Subsystem: Creative Labs: Unknown device 8064
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (500ns min, 5000ns max)
Interrupt: pin A routed to IRQ 10
Region 0: I/O ports at c400 [size=32]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

02:05.1 Input device controller: Creative Labs SB Live! (rev 08)
Subsystem: Creative Labs Gameport Joystick
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32
Region 0: I/O ports at c800 [size=8]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

02:06.0 Multimedia video controller: Brooktree Corporation Bt878 (rev 02)
Subsystem: Hauppauge computer works Inc. WinTV/GO
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (4000ns min, 10000ns max)
Interrupt: pin A routed to IRQ 14
Region 0: Memory at f5800000 (32-bit, prefetchable) [size=4K]

02:06.1 Multimedia controller: Brooktree Corporation Bt878 (rev 02)
Subsystem: Hauppauge computer works Inc. WinTV/GO
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (1000ns min, 63750ns max)
Interrupt: pin A routed to IRQ 14
Region 0: Memory at f5801000 (32-bit, prefetchable) [size=4K]

02:09.0 Ethernet controller: Intel Corp. 82557 [Ethernet Pro 100] (rev 10)
Subsystem: IBM: Unknown device 0207
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32 (2000ns min, 14000ns max), cache line size 08
Interrupt: pin A routed to IRQ 10
Region 0: Memory at f4021000 (32-bit, non-prefetchable) [size=4K]
Region 1: I/O ports at cc00 [size=64]
Region 2: Memory at f4000000 (32-bit, non-prefetchable) [size=128K]
Expansion ROM at <unassigned> [disabled] [size=64K]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=2 PME-

# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 04 Lun: 00
Vendor: PIONEER Model: DVD-ROM DVD-305 Rev: 1.00
Type: CD-ROM ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 05 Lun: 00
Vendor: SEAGATE Model: DAT 04106-XXX Rev: 7550
Type: Sequential-Access ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 06 Lun: 00
Vendor: YAMAHA Model: CRW8424S Rev: 1.0j
Type: CD-ROM ANSI SCSI revision: 02
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: 3ware Model: Logical Disk 0 Rev: 1.0
Type: Direct-Access ANSI SCSI revision: ffffffff