[syzbot] possible deadlock in f2fs_handle_error

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    6d464646530f Merge branch 'for-next/core' into for-kernelci
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=177fa381880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=23eec5c79c22aaf8
dashboard link: https://syzkaller.appspot.com/bug?extid=fbc90a26c932581cfe6b
compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/f22d29413625/disk-6d464646.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/389f0a5f1a4a/vmlinux-6d464646.xz
kernel image: https://storage.googleapis.com/syzbot-assets/48ddb02d82da/Image-6d464646.gz.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+fbc90a26c932581cfe6b@xxxxxxxxxxxxxxxxxxxxxxxxx

loop3: detected capacity change from 0 to 262144
F2FS-fs (loop3): Mismatch valid blocks 0 vs. 1
======================================================
WARNING: possible circular locking dependency detected
6.1.0-rc6-syzkaller-32662-g6d464646530f #0 Not tainted
------------------------------------------------------
syz-executor.3/2078 is trying to acquire lock:
ffff00010e914088 (&sbi->sb_lock){++++}-{3:3}, at: f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
ffff00010e914088 (&sbi->sb_lock){++++}-{3:3}, at: f2fs_handle_error+0x9c/0x17c fs/f2fs/super.c:3898

but task is already holding lock:
ffff0000fd2733b8 (&array[i].journal_rwsem){++++}-{3:3}, at: build_sit_entries+0x568/0x9b8 fs/f2fs/segment.c:4412

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #6 (&array[i].journal_rwsem){++++}-{3:3}:
       down_read+0x5c/0x78 kernel/locking/rwsem.c:1509
       scan_curseg_cache fs/f2fs/node.c:2401 [inline]
       __f2fs_build_free_nids fs/f2fs/node.c:2513 [inline]
       f2fs_build_free_nids+0x798/0x8f4 fs/f2fs/node.c:2528
       f2fs_build_node_manager+0x624/0x64c fs/f2fs/node.c:3313
       f2fs_fill_super+0x1470/0x1e90 fs/f2fs/super.c:4306
       mount_bdev+0x1b8/0x210 fs/super.c:1401
       f2fs_mount+0x44/0x58 fs/f2fs/super.c:4580
       legacy_get_tree+0x30/0x74 fs/fs_context.c:610
       vfs_get_tree+0x40/0x140 fs/super.c:1531
       do_new_mount+0x1dc/0x4e4 fs/namespace.c:3040
       path_mount+0x358/0x890 fs/namespace.c:3370
       do_mount fs/namespace.c:3383 [inline]
       __do_sys_mount fs/namespace.c:3591 [inline]
       __se_sys_mount fs/namespace.c:3568 [inline]
       __arm64_sys_mount+0x2c4/0x3c4 fs/namespace.c:3568
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
       el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
       el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #5 (&nm_i->nat_tree_lock){++++}-{3:3}:
       down_read+0x5c/0x78 kernel/locking/rwsem.c:1509
       f2fs_down_read fs/f2fs/f2fs.h:2180 [inline]
       f2fs_get_node_info+0x74/0x458 fs/f2fs/node.c:560
       f2fs_do_write_data_page+0x638/0x1094 fs/f2fs/data.c:2710
       f2fs_write_single_data_page+0x750/0xb24 fs/f2fs/data.c:2845
       f2fs_write_cache_pages+0x498/0xdd0 fs/f2fs/data.c:3097
       __f2fs_write_data_pages+0x3f0/0x47c fs/f2fs/data.c:3247
       f2fs_write_data_pages+0x44/0x54 fs/f2fs/data.c:3274
       do_writepages+0x144/0x27c mm/page-writeback.c:2469
       __writeback_single_inode+0x64/0x2e4 fs/fs-writeback.c:1587
       writeback_sb_inodes+0x3e4/0x85c fs/fs-writeback.c:1870
       wb_writeback+0x198/0x328 fs/fs-writeback.c:2044
       wb_do_writeback+0xc8/0x384 fs/fs-writeback.c:2187
       wb_workfn+0x70/0x15c fs/fs-writeback.c:2227
       process_one_work+0x2d8/0x504 kernel/workqueue.c:2289
       worker_thread+0x340/0x610 kernel/workqueue.c:2436
       kthread+0x12c/0x158 kernel/kthread.c:376
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:863

-> #4 (&sbi->cp_rwsem){++++}-{3:3}:
       down_read+0x5c/0x78 kernel/locking/rwsem.c:1509
       f2fs_down_read fs/f2fs/f2fs.h:2180 [inline]
       f2fs_lock_op fs/f2fs/f2fs.h:2223 [inline]
       f2fs_do_truncate_blocks+0x98/0x384 fs/f2fs/file.c:687
       f2fs_truncate_blocks+0x9c/0x1d8 fs/f2fs/file.c:750
       f2fs_truncate+0x1b8/0x2a8 fs/f2fs/file.c:802
       f2fs_setattr+0x5c8/0x788 fs/f2fs/file.c:1009
       notify_change+0x758/0x7f0 fs/attr.c:420
       do_truncate+0x108/0x150 fs/open.c:65
       handle_truncate+0xf4/0x154 fs/namei.c:3216
       do_open fs/namei.c:3561 [inline]
       path_openat+0xee0/0x11c4 fs/namei.c:3713
       do_filp_open+0xdc/0x1b8 fs/namei.c:3740
       do_sys_openat2+0xb8/0x22c fs/open.c:1310
       do_sys_open fs/open.c:1326 [inline]
       __do_sys_openat fs/open.c:1342 [inline]
       __se_sys_openat fs/open.c:1337 [inline]
       __arm64_sys_openat+0xb0/0xe0 fs/open.c:1337
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
       el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
       el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #3 (mapping.invalidate_lock#4){++++}-{3:3}:
       down_read+0x5c/0x78 kernel/locking/rwsem.c:1509
       filemap_invalidate_lock_shared include/linux/fs.h:811 [inline]
       f2fs_vm_page_mkwrite+0x18c/0x9a4 fs/f2fs/file.c:104
       do_page_mkwrite+0x74/0x288 mm/memory.c:2978
       wp_page_shared+0x8c/0x364 mm/memory.c:3324
       do_wp_page+0x794/0xd10 mm/memory.c:3474
       handle_pte_fault mm/memory.c:4973 [inline]
       __handle_mm_fault mm/memory.c:5097 [inline]
       handle_mm_fault+0x880/0xa48 mm/memory.c:5218
       __do_page_fault arch/arm64/mm/fault.c:512 [inline]
       do_page_fault+0x428/0x79c arch/arm64/mm/fault.c:612
       do_mem_abort+0x54/0x130 arch/arm64/mm/fault.c:831
       el0_da+0x70/0x16c arch/arm64/kernel/entry-common.c:515
       el0t_64_sync_handler+0xcc/0xf0 arch/arm64/kernel/entry-common.c:658
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #2 (sb_pagefaults#2){.+.+}-{0:0}:
       percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
       __sb_start_write+0x84/0x1f4 include/linux/fs.h:1826
       sb_start_pagefault include/linux/fs.h:1930 [inline]
       f2fs_vm_page_mkwrite+0x160/0x9a4 fs/f2fs/file.c:99
       do_page_mkwrite+0x74/0x288 mm/memory.c:2978
       wp_page_shared+0x8c/0x364 mm/memory.c:3324
       do_wp_page+0x794/0xd10 mm/memory.c:3474
       handle_pte_fault mm/memory.c:4973 [inline]
       __handle_mm_fault mm/memory.c:5097 [inline]
       handle_mm_fault+0x880/0xa48 mm/memory.c:5218
       __do_page_fault arch/arm64/mm/fault.c:512 [inline]
       do_page_fault+0x428/0x79c arch/arm64/mm/fault.c:612
       do_mem_abort+0x54/0x130 arch/arm64/mm/fault.c:831
       el0_da+0x70/0x16c arch/arm64/kernel/entry-common.c:515
       el0t_64_sync_handler+0xcc/0xf0 arch/arm64/kernel/entry-common.c:658
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #1 (&mm->mmap_lock){++++}-{3:3}:
       __might_fault+0x7c/0xb4 mm/memory.c:5646
       _copy_to_user include/linux/uaccess.h:143 [inline]
       copy_to_user include/linux/uaccess.h:169 [inline]
       f2fs_ioc_get_encryption_pwsalt fs/f2fs/file.c:2349 [inline]
       __f2fs_ioctl+0x3204/0x3318 fs/f2fs/file.c:4151
       f2fs_ioctl+0x74/0xbc fs/f2fs/file.c:4224
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:870 [inline]
       __se_sys_ioctl fs/ioctl.c:856 [inline]
       __arm64_sys_ioctl+0xd0/0x140 fs/ioctl.c:856
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
       el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
       el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #0 (&sbi->sb_lock){++++}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3097 [inline]
       check_prevs_add kernel/locking/lockdep.c:3216 [inline]
       validate_chain kernel/locking/lockdep.c:3831 [inline]
       __lock_acquire+0x1530/0x3084 kernel/locking/lockdep.c:5055
       lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5668
       down_write+0x5c/0x88 kernel/locking/rwsem.c:1562
       f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
       f2fs_handle_error+0x9c/0x17c fs/f2fs/super.c:3898
       check_block_count+0x210/0x268
       build_sit_entries+0x638/0x9b8 fs/f2fs/segment.c:4432
       f2fs_build_segment_manager+0x354/0x408 fs/f2fs/segment.c:5123
       f2fs_fill_super+0x1438/0x1e90 fs/f2fs/super.c:4300
       mount_bdev+0x1b8/0x210 fs/super.c:1401
       f2fs_mount+0x44/0x58 fs/f2fs/super.c:4580
       legacy_get_tree+0x30/0x74 fs/fs_context.c:610
       vfs_get_tree+0x40/0x140 fs/super.c:1531
       do_new_mount+0x1dc/0x4e4 fs/namespace.c:3040
       path_mount+0x358/0x890 fs/namespace.c:3370
       do_mount fs/namespace.c:3383 [inline]
       __do_sys_mount fs/namespace.c:3591 [inline]
       __se_sys_mount fs/namespace.c:3568 [inline]
       __arm64_sys_mount+0x2c4/0x3c4 fs/namespace.c:3568
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
       el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
       el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

other info that might help us debug this:

Chain exists of:
  &sbi->sb_lock --> &nm_i->nat_tree_lock --> &array[i].journal_rwsem

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&array[i].journal_rwsem);
                               lock(&nm_i->nat_tree_lock);
                               lock(&array[i].journal_rwsem);
  lock(&sbi->sb_lock);

 *** DEADLOCK ***

2 locks held by syz-executor.3/2078:
 #0: ffff000144b030e0 (&type->s_umount_key#53/1){+.+.}-{3:3}, at: alloc_super+0xf8/0x430 fs/super.c:228
 #1: ffff0000fd2733b8 (&array[i].journal_rwsem){++++}-{3:3}, at: build_sit_entries+0x568/0x9b8 fs/f2fs/segment.c:4412

stack backtrace:
CPU: 0 PID: 2078 Comm: syz-executor.3 Not tainted 6.1.0-rc6-syzkaller-32662-g6d464646530f #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/30/2022
Call trace:
 dump_backtrace+0x1c4/0x1f0 arch/arm64/kernel/stacktrace.c:156
 show_stack+0x2c/0x54 arch/arm64/kernel/stacktrace.c:163
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x104/0x16c lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x2c4/0x2c8 kernel/locking/lockdep.c:2055
 check_noncircular+0x14c/0x154 kernel/locking/lockdep.c:2177
 check_prev_add kernel/locking/lockdep.c:3097 [inline]
 check_prevs_add kernel/locking/lockdep.c:3216 [inline]
 validate_chain kernel/locking/lockdep.c:3831 [inline]
 __lock_acquire+0x1530/0x3084 kernel/locking/lockdep.c:5055
 lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5668
 down_write+0x5c/0x88 kernel/locking/rwsem.c:1562
 f2fs_down_write fs/f2fs/f2fs.h:2205 [inline]
 f2fs_handle_error+0x9c/0x17c fs/f2fs/super.c:3898
 check_block_count+0x210/0x268
 build_sit_entries+0x638/0x9b8 fs/f2fs/segment.c:4432
 f2fs_build_segment_manager+0x354/0x408 fs/f2fs/segment.c:5123
 f2fs_fill_super+0x1438/0x1e90 fs/f2fs/super.c:4300
 mount_bdev+0x1b8/0x210 fs/super.c:1401
 f2fs_mount+0x44/0x58 fs/f2fs/super.c:4580
 legacy_get_tree+0x30/0x74 fs/fs_context.c:610
 vfs_get_tree+0x40/0x140 fs/super.c:1531
 do_new_mount+0x1dc/0x4e4 fs/namespace.c:3040
 path_mount+0x358/0x890 fs/namespace.c:3370
 do_mount fs/namespace.c:3383 [inline]
 __do_sys_mount fs/namespace.c:3591 [inline]
 __se_sys_mount fs/namespace.c:3568 [inline]
 __arm64_sys_mount+0x2c4/0x3c4 fs/namespace.c:3568
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
 el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
 el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
 el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584
F2FS-fs (loop3): Failed to initialize F2FS segment manager (-117)


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@xxxxxxxxxxxxxxxx.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.