Re: Question about btrfs no space left error and forced readonly
From: Qu Wenruo
Date: Thu Jan 18 2024 - 02:08:31 EST
On 2024/1/17 18:17, ChenXiaoSong wrote:
Greetings,
We had a btrfs no space left error and forced readonly, the stack trace
is [1]. This happened on our kernel release version based on lts 4.19.
Is there already a fix patch for this issue?
Or can anybody please advise on how to debug this further?
Full dmesg please (if there is any one before the call trace).
And `btrfs fi usage`, `btrfs fi df` output, along with `btrfs check
--readonly` if possible.
Thanks,
Qu
Thanks,
ChenXiaoSong.
[1]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328533] ------------[
cut here ]------------
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328594] WARNING: CPU:
42 PID: 460094 at fs/btrfs/extent-tree.c:6805
__btrfs_free_extent.isra.71+0x65c/0xbd0 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328595] Modules linked
in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace
fscache fuse xt_nat veth nf_conntrack_netlink xt_conntrack br_netfilter
bridge sch_htb xt_addrtype xt_set ipt_MASQUERADE xt_mark
ip_set_hash_ipportnet ip_set_bitmap_port ip_set_hash_ipportip
ip_set_hash_ipport dummy xt_comment iptable_nat nf_nat_ipv4 nf_nat
iptable_filter nls_utf8 isofs bonding 8021q garp mrp stp llc ip_set
nfnetlink ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 sunrpc amd64_edac_mod edac_mce_amd btrfs xor
zstd_decompress zstd_compress raid6_pq libcrc32c kvm_amd ccp kvm
irqbypass ipmi_ssif ast ttm drm_kms_helper syscopyarea sysfillrect igb
sysimgblt i2c_algo_bit fb_sys_fops pcspkr txgbe sg dca drm i2c_piix4
k10temp ipmi_si ipmi_devintf ipmi_msghandler binfmt_misc
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328634] ip_tables
ext4 mbcache jbd2 sd_mod crc32c_intel ahci libahci megaraid_sas libata
dm_mirror dm_region_hash dm_log dm_mod
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328647] CPU: 42 PID:
460094 Comm: httpWorkerThrea Kdump: loaded Tainted: G W
4.19.90-17.ky10.x86_64 #1
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328648] Hardware name:
Sugon H620-G30/65N32-US, BIOS 0SSSX245 06/07/2020
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328656] RIP:
0010:__btrfs_free_extent.isra.71+0x65c/0xbd0 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328659] Code: 44 89 e9
ba c8 1a 00 00 48 c7 c6 a0 aa 06 c1 e8 fc ba 09 00 e9 fe fa ff ff 0f 0b
44 89 ee 48 c7 c7 68 7c 07 c1 e8 14 e9 2d e5 <0f> 0b e9 ce fa ff ff 41
83 f8 29 0f 86 d6 01 00 00 48 8b 7c 24 08
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328660] RSP:
0018:ffffae7f9f6e78c8 EFLAGS: 00010286
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328661] RAX:
0000000000000000 RBX: 000000000000a0c9 RCX: 0000000000000006
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328662] RDX:
0000000000000007 RSI: 0000000000000092 RDI: ffff90a4ffa96850
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328663] RBP:
00000470931db000 R08: 000000000210773e R09: 0000000000000004
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328664] R10:
00000000ffffffe4 R11: 0000000000000001 R12: ffff909b302e2a80
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328664] R13:
00000000ffffffe4 R14: 0000000000000000 R15: 000000000003fc62
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328666] FS:
00007f4aedae2700(0000) GS:ffff90a4ffa80000(0000) knlGS:0000000000000000
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328667] CS: 0010 DS:
0000 ES: 0000 CR0: 0000000080050033
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328667] CR2:
00007efce4073ff0 CR3: 0000006413d3c000 CR4: 00000000003406e0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328668] Call Trace:
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328682]
__btrfs_run_delayed_refs+0x4f8/0x1150 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328707]
btrfs_run_delayed_refs+0xe7/0x1b0 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328717]
btrfs_truncate_inode_items+0xa56/0xeb0 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328728]
btrfs_evict_inode+0x496/0x4f0 [btrfs]
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328733] evict+0xd2/0x1a0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328736]
__dentry_kill+0xdd/0x180
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328738]
dentry_kill+0x4d/0x260
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328740] dput+0x183/0x200
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328743]
__fput+0x118/0x1f0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328747]
task_work_run+0x8a/0xb0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328750]
do_exit+0x2ec/0xbf0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328753]
do_group_exit+0x3a/0xa0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328755]
get_signal+0x13f/0x7a0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328758]
do_signal+0x36/0x610
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328768]
exit_to_usermode_loop+0x71/0xe0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328770]
do_syscall_64+0x1a3/0x1d0
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328773]
entry_SYSCALL_64_after_hwframe+0x44/0xa9
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328779] RIP:
0033:0x7f4b12fae10c
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328785] Code: Bad RIP
value.
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328785] RSP:
002b:00007f4aedae1550 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328786] RAX:
fffffffffffffe00 RBX: 00007f4b0d1914e0 RCX: 00007f4b12fae10c
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328787] RDX:
0000000000000000 RSI: 0000000000000080 RDI: 00007f4b0d19150c
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328787] RBP:
0000000000000000 R08: 0000000000000000 R09: 0000000794039ff8
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328788] R10:
0000000000000000 R11: 0000000000000246 R12: 00007f4b0d1914b8
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328789] R13:
0000000000008223 R14: 0000000000000000 R15: 00007f4b0d19150c
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328790] ---[ end trace
74a7f2461a0f9473 ]---
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.328961] BTRFS: error
(device dm-9) in __btrfs_free_extent:6805: errno=-28 No space left
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.439057] BTRFS info
(device dm-9): forced readonly
Dec 8 15:25:17 pjdhcpaasnap8pn kernel: [13439425.439067] BTRFS: error
(device dm-9) in btrfs_run_delayed_refs:2935: errno=-28 No space left