Possible 2.6.23 regression - Disappearing disk space

From: James Ausmus
Date: Wed Oct 24 2007 - 14:05:46 EST


Since updating my laptop to 2.6.23, occasionally all of my free disk
space on my root partition will just go away, with no files accounting
for the space, with no odd messages in dmesg or my syslog. If I
reboot, I immediately have the proper amount of free space again. Here
is the output of a du -sx * on /, and the output of the df command
when the problem is occuring, followed by the same info after a fresh
reboot (literally just did the command in the failed state, then
immediately rebooted and ran the same commands again) - any thoughts
as to what might be happening?

As a note - when I first see the issue, I have exactly 0 free space
available on root, as per df - I then delete some random things in
order to have enough free space to operate, which is why in my first
df you see 55M available

Prior to reboot (with issue occurring):

charles / # du -sx *
8814 bin
0 boot
132 dev
392 emul
105869 etc
1718345 home
0 lib
4858 lib32
72358 lib64
4 media
72374 mnt
2027054 opt
0 proc
126411 root
15596 sbin
0 sys
28237 tmp
20887995 usr
1098215 var

charles / # df

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda4 37020624 36964772 55852 100% /
udev 10240 132 10108 2% /dev
shm 1028712 24 1028688 1% /dev/shm
/dev/sdc2 73246080 20255824 52990256 28% /media/linux

charles / #

du -sx totals: 26,166,654

Post reboot:

charles / # df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda4 37020624 24449440 12571184 67% /
udev 10240 132 10108 2% /dev
shm 1028712 0 1028712 0% /dev/shm
/dev/sdg2 73246080 20255824 52990256 28% /media/linux
charles / # du -sx /

charles / # du -sx *
8814 bin
0 boot
132 dev
392 emul
105869 etc
1719241 home
0 lib
4858 lib32
72358 lib64
4 media
72374 mnt
2027054 opt
0 proc
126406 root
15596 sbin
0 sys
28249 tmp
20887995 usr
1040963 var

charles / # df

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda4 37020624 24452848 12567776 67% /
udev 10240 132 10108 2% /dev
shm 1028712 24 1028688 1% /dev/shm
/dev/sdg2 73246080 20255824 52990256 28% /media/linux

charles / #

charles / # uname -a
Linux charles 2.6.23-gentoo #1 SMP PREEMPT Tue Oct 16 11:34:08 PDT
2007 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4400+
AuthenticAMD GNU/Linux

charles / # mount
/dev/sda4 on / type reiserfs (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw,nosuid,noexec)
shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)
usbfs on /proc/bus/usb type usbfs (rw,noexec,nosuid,devmode=0664,devgid=85)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,noexec,nosuid,nodev)
nfsd on /proc/fs/nfs type nfsd (rw,noexec,nosuid,nodev)
/dev/sdg2 on /media/linux type reiserfs (rw,nosuid,nodev)
/media/linux/usr/portage/distfiles on /usr/portage/distfiles type none
(rw,noexec,nosuid,nodev,bind)



As you can see, the du -sx * gives virtually identical used space
(which adds up to about 26G, in both instances), but df thinks that
there is no free space available, and I can't write anything to disk
(I get a disk full error).

Attached is output from lspci -v, and zcat /proc/config.gz

Thanks!

-James

Attachment: config
Description: Binary data

00:00.0 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, 66MHz, medium devsel, latency 16
Memory at <ignored> (32-bit, prefetchable)
Capabilities: [80] AGP version 3.0
Capabilities: [50] Power Management version 2
Capabilities: [60] HyperTransport: Slave or Primary Interface
Capabilities: [58] HyperTransport: Interrupt Discovery and Configuration

00:00.1 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 0

00:00.2 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 0

00:00.3 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 0

00:00.4 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 0

00:00.5 PIC: VIA Technologies, Inc. K8T890 I/O APIC Interrupt Controller (prog-if 20 [IO(X)-APIC])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, fast devsel, latency 0

00:00.7 Host bridge: VIA Technologies, Inc. K8T890 Host Bridge
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 0

00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI bridge [K8T800/K8T890 South] (prog-if 00 [Normal decode])
Flags: bus master, 66MHz, medium devsel, latency 0
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
Capabilities: [70] Power Management version 2

00:02.0 PCI bridge: VIA Technologies, Inc. K8T890 PCI to PCI Bridge Controller (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 00005000-00005fff
Memory behind bridge: d0000000-d1ffffff
Prefetchable memory behind bridge: c0000000-cfffffff
Capabilities: [40] Express Root Port (Slot-), MSI 14
Capabilities: [68] Power Management version 2
Capabilities: [70] Message Signalled Interrupts: Mask+ 64bit+ Queue=0/0 Enable+
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel

00:03.0 PCI bridge: VIA Technologies, Inc. K8T890 PCI to PCI Bridge Controller (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
Capabilities: [40] Express Root Port (Slot-), MSI 14
Capabilities: [68] Power Management version 2
Capabilities: [70] Message Signalled Interrupts: Mask+ 64bit+ Queue=0/0 Enable+
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel

00:03.2 PCI bridge: VIA Technologies, Inc. K8T890 PCI to PCI Bridge Controller (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
Capabilities: [40] Express Root Port (Slot-), MSI 14
Capabilities: [68] Power Management version 2
Capabilities: [70] Message Signalled Interrupts: Mask+ 64bit+ Queue=0/0 Enable+
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Virtual Channel

00:05.0 CardBus bridge: Texas Instruments PCI1410 PC card Cardbus Controller (rev 02)
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 168, IRQ 18
Memory at 90020000 (32-bit, non-prefetchable) [size=4K]
Bus: primary=00, secondary=05, subordinate=08, sec-latency=176
Memory window 0: 88000000-8bfff000 (prefetchable)
Memory window 1: 8c000000-8ffff000
I/O window 0: 00002400-000024ff
I/O window 1: 00002800-000028ff
16-bit legacy interface ports at 0001

00:06.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link) (prog-if 10 [OHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: medium devsel, IRQ 11
Memory at d2206000 (32-bit, non-prefetchable) [size=2K]
Memory at d2200000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 2

00:08.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
I/O ports at 1000 [size=256]
Memory at d2206800 (32-bit, non-prefetchable) [size=256]
[virtual] Expansion ROM at 90000000 [disabled] [size=128K]
Capabilities: [dc] Power Management version 2

00:0a.0 Network controller: RaLink RT2500 802.11g Cardbus/mini-PCI (rev 01)
Subsystem: Micro-Star International Co., Ltd. Unknown 802.11g mini-PCI Adapter
Flags: bus master, slow devsel, latency 64, IRQ 17
Memory at d2204000 (32-bit, non-prefetchable) [size=8K]
Capabilities: [40] Power Management version 2

00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 20
I/O ports at 20b0 [size=8]
I/O ports at 20a4 [size=4]
I/O ports at 20a8 [size=8]
I/O ports at 20a0 [size=4]
I/O ports at 2080 [size=16]
I/O ports at 1400 [size=256]
Capabilities: [c0] Power Management version 2

00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06) (prog-if 8a [Master SecP PriP])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 20
[virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
[virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
I/O ports at 2090 [size=16]
Capabilities: [c0] Power Management version 2

00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81) (prog-if 00 [UHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 21
I/O ports at 2000 [size=32]
Capabilities: [80] Power Management version 2

00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81) (prog-if 00 [UHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 21
I/O ports at 2020 [size=32]
Capabilities: [80] Power Management version 2

00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81) (prog-if 00 [UHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 21
I/O ports at 2040 [size=32]
Capabilities: [80] Power Management version 2

00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81) (prog-if 00 [UHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 21
I/O ports at 2060 [size=32]
Capabilities: [80] Power Management version 2

00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86) (prog-if 20 [EHCI])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, medium devsel, latency 64, IRQ 21
Memory at d2206c00 (32-bit, non-prefetchable) [size=256]
Capabilities: [80] Power Management version 2

00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [KT600/K8T800/K8T890 South]
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, stepping, medium devsel, latency 0
Capabilities: [c0] Power Management version 2

00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 60)
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: medium devsel, IRQ 22
I/O ports at 1800 [size=256]
Capabilities: [c0] Power Management version 2

00:11.6 Communication controller: VIA Technologies, Inc. AC'97 Modem Controller (rev 80)
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: medium devsel, IRQ 22
I/O ports at 1c00 [size=256]
Capabilities: [d0] Power Management version 2

00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
Flags: fast devsel
Capabilities: [80] HyperTransport: Host or Secondary Interface

00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
Flags: fast devsel

00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
Flags: fast devsel

00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
Flags: fast devsel

02:00.0 VGA compatible controller: nVidia Corporation G70 [GeForce Go 7800 GTX] (rev a1) (prog-if 00 [VGA])
Subsystem: CLEVO/KAPOK Computer Unknown device 0901
Flags: bus master, fast devsel, latency 0, IRQ 24
Memory at d1000000 (32-bit, non-prefetchable) [size=16M]
Memory at c0000000 (64-bit, prefetchable) [size=256M]
Memory at d0000000 (64-bit, non-prefetchable) [size=16M]
I/O ports at 5000 [size=128]
Capabilities: [60] Power Management version 2
Capabilities: [68] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [128] Power Budgeting