[GIT PULL] Btrfs updates for 6.5
From: David Sterba
Date: Mon Jun 26 2023 - 14:22:42 EST
Hi,
there are mainly core changes, refactoring and optimizations.
Performance is improved in some areas, overall there may be a cumulative
improvement due to refactoring that removed lookups in the IO path or
simplified IO submission tracking.
No merge conflicts. Please pull, thanks.
Core:
- submit IO synchronously for fast checksums (crc32c and xxhash), remove
high priority worker kthread
- read extent buffer in one go, simplify IO tracking, bio submission and
locking
- remove additional tracking of redirtied extent buffers, originally
added for zoned mode but actually not needed
- track ordered extent pointer in bio to avoid rbtree lookups during IO
- scrub, use recovered data stripes as cache to avoid unnecessary read
- in zoned mode, optimize logical to physical mappings of extents
- remove PageError handling, not set by VFS nor writeback
- cleanups, refactoring, better structure packing
- lots of error handling improvements
- more assertions, lockdep annotations
- print assertion failure with the exact line where it happens
- tracepoint updates
- more debugging prints
Performance:
- speedup in fsync(), better tracking of inode logged status can avoid
transaction commit
- IO path structures track logical offsets in data structures and does
not need to look it up
User visible changes:
- don't commit transaction for every created subvolume, this can reduce
time when many subvolumes are created in a batch
- print affected files when relocation fails
- trigger orphan file cleanup during START_SYNC ioctl
Notable fixes:
- fix crash when disabling quota and relocation
- fix crashes when removing roots from drity list
- fix transacion abort during relocation when converting from newer
profiles not covered by fallback
- in zoned mode, stop reclaiming block groups if filesystem becomes
read-only
- fix rare race condition in tree mod log rewind that can miss some
btree node slots
- with enabled fsverity, drop up-to-date page bit in case the
verification fails
----------------------------------------------------------------
The following changes since commit 45a3e24f65e90a047bef86f927ebdc4c710edaa1:
Linux 6.4-rc7 (2023-06-18 14:06:27 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-6.5
for you to fetch changes up to 8a4a0b2a3eaf75ca8854f856ef29690c12b2f531:
btrfs: fix race between quota disable and relocation (2023-06-19 20:29:25 +0200)
----------------------------------------------------------------
Anand Jain (11):
btrfs: use SECTOR_SHIFT to convert physical offset to LBA
btrfs: use SECTOR_SHIFT to convert LBA to physical offset
btrfs: reduce struct btrfs_fs_devices size by moving fsid_change
btrfs: streamline fsid checks in alloc_fs_devices
btrfs: merge calls to alloc_fs_devices in device_list_add
btrfs: add comment about metadata_uuid in btrfs_fs_devices
btrfs: return bool from check_tree_block_fsid instead of int
btrfs: simplify fsid and metadata_uuid comparisons
btrfs: simplify how changed fsid and metadata_uuid is checked
btrfs: consolidate uuid comparisons in btrfs_validate_super
btrfs: add and fix comments in btrfs_fs_devices
Ben Dooks (1):
btrfs: make btrfs_compressed_bioset static
Boris Burkov (2):
btrfs: warn on invalid slot in tree mod log rewind
btrfs: insert tree mod log move in push_node_left
Christoph Hellwig (80):
btrfs: submit IO synchronously for fast checksum implementations
btrfs: determine synchronous writers from bio or writeback control
btrfs: remove hipri_workers workqueue
btrfs: fix dirty_metadata_bytes for redirtied buffers
btrfs: don't hold an extra reference for redirtied buffers
btrfs: mark extent_buffer_under_io static
btrfs: subpage: fix error handling in end_bio_subpage_eb_writepage
btrfs: move setting the buffer uptodate out of validate_extent_buffer
btrfs: merge verify_parent_transid and btrfs_buffer_uptodate
btrfs: always read the entire extent_buffer
btrfs: don't use btrfs_bio_ctrl for extent buffer reading
btrfs: remove the mirror_num argument to btrfs_submit_compressed_read
btrfs: use a separate end_io handler for read_extent_buffer
btrfs: do not try to unlock the extent for non-subpage metadata reads
btrfs: return bool from lock_extent_buffer_for_io
btrfs: submit a writeback bio per extent_buffer
btrfs: move page locking from lock_extent_buffer_for_io to write_one_eb
btrfs: don't use btrfs_bio_ctrl for extent buffer writing
btrfs: use a separate end_io handler for extent_buffer writing
btrfs: remove the extent_buffer lookup in btree block checksumming
btrfs: remove the io_pages field in struct extent_buffer
btrfs: stop using PageError for extent_buffers
btrfs: don't check for uptodate pages in read_extent_buffer_pages
btrfs: stop using lock_extent in btrfs_buffer_uptodate
btrfs: use per-buffer locking for extent_buffer reading
btrfs: merge write_one_subpage_eb into write_one_eb
btrfs: fix the btrfs_get_global_root return value
btrfs: convert btrfs_get_global_root to use a switch statement
btrfs: remove a pointless NULL check in btrfs_lookup_fs_root
btrfs: optimize out btrfs_is_zoned for !CONFIG_BLK_DEV_ZONED
btrfs: don't call btrfs_record_physical_zoned for failed append
btrfs: mark the len field in struct btrfs_ordered_sum as unsigned
btrfs: rename the bytenr field in struct btrfs_ordered_sum to logical
btrfs: optimize the logical to physical mapping for zoned writes
btrfs: record orig_physical only for the original bio
btrfs: move split_extent_map to extent_map.c
btrfs: reorder conditions in btrfs_extract_ordered_extent
btrfs: return the new ordered_extent from btrfs_split_ordered_extent
btrfs: split btrfs_alloc_ordered_extent to allocation and insertion helpers
btrfs: atomically insert the new extent in btrfs_split_ordered_extent
btrfs: handle completed ordered extents in btrfs_split_ordered_extent
btrfs: defer splitting of ordered extents until I/O completion
btrfs: pass the new logical address to split_extent_map
btrfs: remove unused BTRFS_MAP_DISCARD
btrfs: optimize simple reads in btrfsic_map_block
btrfs: remove unused btrfs_map_block
btrfs: rename __btrfs_map_block to btrfs_map_block
btrfs: open code btrfs_map_sblock
btrfs: open code need_full_stripe conditions
btrfs: fix range_end calculation in extent_write_locked_range
btrfs: factor out a btrfs_verify_page helper
btrfs: fix fsverify read error handling in end_page_read
btrfs: don't check PageError in btrfs_verify_page
btrfs: don't fail writeback when allocating the compression context fails
btrfs: rename cow_file_range_async to run_delalloc_compressed
btrfs: don't check PageError in __extent_writepage
btrfs: stop setting PageError in the data I/O path
btrfs: remove PAGE_SET_ERROR
btrfs: remove non-standard extent handling in __extent_writepage_io
btrfs: move writeback_control::nr_to_write update to __extent_writepage
btrfs: only call __extent_writepage_io from extent_write_locked_range
btrfs: don't treat zoned writeback as being from an async helper thread
btrfs: fix file_offset for REQ_BTRFS_ONE_ORDERED bios that get split
btrfs: limit write bios to a single ordered extent
btrfs: merge the two calls to btrfs_add_ordered_extent in run_delalloc_nocow
btrfs: pass an ordered_extent to btrfs_reloc_clone_csums
btrfs: pass an ordered_extent to btrfs_submit_compressed_write
btrfs: remove btrfs_add_ordered_extent
btrfs: add a is_data_bbio helper
btrfs: open code btrfs_bio_end_io in btrfs_dio_submit_io
btrfs: add an ordered_extent pointer to struct btrfs_bio
btrfs: use bbio->ordered in btrfs_csum_one_bio
btrfs: factor out a can_finish_ordered_extent helper
btrfs: factor out a btrfs_queue_ordered_fn helper
btrfs: add a btrfs_finish_ordered_extent helper
btrfs: open code end_extent_writepage in end_bio_extent_writepage
btrfs: use btrfs_finish_ordered_extent to complete compressed writes
btrfs: use btrfs_finish_ordered_extent to complete direct writes
btrfs: use btrfs_finish_ordered_extent to complete buffered writes
btrfs: set FMODE_CAN_ODIRECT instead of a dummy direct_IO method
David Sterba (13):
btrfs: open code set_extent_defrag
btrfs: open code set_extent_delalloc
btrfs: open code set_extent_new
btrfs: open code set_extent_dirty
btrfs: open code set_extent_bits_nowait
btrfs: open code set_extent_bits
btrfs: drop NOFAIL from set_extent_bit allocation masks
btrfs: pass NOWAIT for set/clear extent bits as another bit
btrfs: drop gfp from parameter extent state helpers
btrfs: print assertion failure report and stack trace from the same line
btrfs: add xxhash to fast checksum implementations
btrfs: disable allocation warnings for compression workspaces
btrfs: add block-group tree to lockdep classes
Filipe Manana (51):
btrfs: unexport btrfs_prev_leaf()
btrfs: make btrfs_free_device() static
btrfs: tag as unlikely the key comparison when checking sibling keys
btrfs: avoid extra memory allocation when copying free space cache
btrfs: avoid searching twice for previous node when merging free space entries
btrfs: use precomputed end offsets at do_trimming()
btrfs: simplify arguments to tree_insert_offset()
btrfs: assert proper locks are held at tree_insert_offset()
btrfs: assert tree lock is held when searching for free space entries
btrfs: assert tree lock is held when linking free space
btrfs: assert tree lock is held when removing free space entries
btrfs: use inode_logged() at need_log_inode()
btrfs: use inode_logged() at btrfs_record_unlink_dir()
btrfs: update comments at btrfs_record_unlink_dir() to be more clear
btrfs: remove pointless label and goto at btrfs_record_unlink_dir()
btrfs: change for_rename argument of btrfs_record_unlink_dir() to bool
btrfs: fix comment referring to no longer existing btrfs_clean_tree_block()
btrfs: reorder some members of struct btrfs_delayed_ref_head
btrfs: remove unused is_head field from struct btrfs_delayed_ref_node
btrfs: remove pointless in_tree field from struct btrfs_delayed_ref_node
btrfs: use a bool to track qgroup record insertion when adding ref head
btrfs: make insert_delayed_ref() return a bool instead of an int
btrfs: get rid of label and goto at insert_delayed_ref()
btrfs: assert correct lock is held at btrfs_select_ref_head()
btrfs: use bool type for delayed ref head fields that are used as booleans
btrfs: use a single switch statement when initializing delayed ref head
btrfs: remove unnecessary prototype declarations at disk-io.c
btrfs: make btrfs_destroy_delayed_refs() return void
btrfs: update documentation for a block group's bg_list member
btrfs: add missing error handling when logging operation while COWing extent buffer
btrfs: fix extent buffer leak after tree mod log failure at split_node()
btrfs: avoid tree mod log ENOMEM failures when we don't need to log
btrfs: do not BUG_ON() on tree mod log failure at __btrfs_cow_block()
btrfs: do not BUG_ON() on tree mod log failure at balance_level()
btrfs: rename enospc label to out at balance_level()
btrfs: avoid unnecessarily setting the fs to RO and error state at balance_level()
btrfs: abort transaction at balance_level() when left child is missing
btrfs: abort transaction at update_ref_for_cow() when ref count is zero
btrfs: do not BUG_ON() on tree mod log failures at push_nodes_for_insert()
btrfs: do not BUG_ON() on tree mod log failure at insert_new_root()
btrfs: do not BUG_ON() on tree mod log failures at insert_ptr()
btrfs: do not BUG_ON() on tree mod log failures at btrfs_del_ptr()
btrfs: replace BUG_ON() at split_item() with proper error handling
btrfs: do not BUG_ON() when dropping inode items from log root
btrfs: send: do not BUG_ON() on unexpected symlink data extent
btrfs: do not BUG_ON on failure to get dir index for new snapshot
btrfs: do not BUG_ON after failure to migrate space during truncation
btrfs: fix race when deleting quota root from the dirty cow roots list
btrfs: fix race when deleting free space root from the dirty cow roots list
btrfs: add comment to struct btrfs_fs_info::dirty_cowonly_roots
btrfs: fix race between quota disable and relocation
Jeff Layton (1):
btrfs: update i_version in update_dev_time
Jiapeng Chong (1):
btrfs: scrub: remove more unused functions
Johannes Thumshirn (1):
btrfs: unexport btrfs_run_discard_work and make it static
Josef Bacik (12):
btrfs: move btrfs_check_trunc_cache_free_space into block-rsv.c
btrfs: remove level argument from btrfs_set_block_flags
btrfs: simplify btrfs_check_leaf_* helpers into a single helper
btrfs: add btrfs_tree_block_status definitions to tree-checker.h
btrfs: use btrfs_tree_block_status for leaf item errors
btrfs: extend btrfs_leaf_check to return btrfs_tree_block_status
btrfs: add __btrfs_check_node helper
btrfs: move btrfs_verify_level_key into tree-checker.c
btrfs: move split_flags/combine_flags helpers to inode-item.h
btrfs: add __KERNEL__ check for btrfs_no_printk
btrfs: add a btrfs_csum_type_size helper
btrfs: rename del_ptr to btrfs_del_ptr and export it
Matt Corallo (1):
btrfs: add handling for RAID1C23/DUP to btrfs_reduce_alloc_profile
Naohiro Aota (6):
btrfs: export bitmap_test_range_all_{set,zero}
btrfs: delete unused BGs while reclaiming BGs
btrfs: move out now unused BG from the reclaim list
btrfs: bail out reclaim process if filesystem is read-only
btrfs: reinsert BGs failed to reclaim
btrfs: tracepoints: also show actual number of the outstanding extents
Qu Wenruo (11):
btrfs: print-tree: pass const extent buffer pointer
btrfs: improve leaf dump and error handling
btrfs: output affected files when relocation fails
btrfs: scrub: use recovered data stripes as cache to avoid unnecessary read
btrfs: handle tree backref walk error properly
btrfs: trigger orphan inode cleanup during START_SYNC ioctl
btrfs: subpage: dump extra subpage bitmaps for debug
btrfs: subpage: make alloc_extent_buffer() handle previously uptodate range efficiently
btrfs: use the same uptodate variable for end_bio_extent_readpage()
btrfs: scrub: remove scrub_ctx::csum_list member
btrfs: scrub: remove btrfs_fs_info::scrub_wr_completion_workers
Sweet Tea Dorminy (1):
btrfs: don't commit transaction for every subvol create
Tejun Heo (1):
btrfs: use alloc_ordered_workqueue() to create ordered workqueues
Tom Rix (1):
btrfs: simplify transid initialization in btrfs_ioctl_wait_sync
fs/btrfs/async-thread.c | 44 +-
fs/btrfs/async-thread.h | 3 +
fs/btrfs/bio.c | 122 +++---
fs/btrfs/bio.h | 29 +-
fs/btrfs/block-group.c | 47 ++-
fs/btrfs/block-group.h | 9 +-
fs/btrfs/block-rsv.c | 19 +
fs/btrfs/block-rsv.h | 2 +
fs/btrfs/btrfs_inode.h | 23 +-
fs/btrfs/check-integrity.c | 21 +-
fs/btrfs/compression.c | 50 +--
fs/btrfs/compression.h | 7 +-
fs/btrfs/ctree.c | 429 +++++++++++++-------
fs/btrfs/ctree.h | 6 +-
fs/btrfs/defrag.c | 3 +-
fs/btrfs/delayed-ref.c | 110 ++---
fs/btrfs/delayed-ref.h | 25 +-
fs/btrfs/dev-replace.c | 6 +-
fs/btrfs/discard.c | 34 +-
fs/btrfs/discard.h | 1 -
fs/btrfs/disk-io.c | 451 ++++-----------------
fs/btrfs/disk-io.h | 7 +-
fs/btrfs/extent-io-tree.c | 37 +-
fs/btrfs/extent-io-tree.h | 62 +--
fs/btrfs/extent-tree.c | 186 ++++-----
fs/btrfs/extent-tree.h | 2 +-
fs/btrfs/extent_io.c | 848 +++++++++++----------------------------
fs/btrfs/extent_io.h | 11 +-
fs/btrfs/extent_map.c | 110 ++++-
fs/btrfs/extent_map.h | 6 +-
fs/btrfs/file-item.c | 90 ++---
fs/btrfs/file-item.h | 1 +
fs/btrfs/file.c | 12 +-
fs/btrfs/free-space-cache.c | 122 +++---
fs/btrfs/free-space-cache.h | 2 -
fs/btrfs/free-space-tree.c | 3 +
fs/btrfs/fs.h | 5 +-
fs/btrfs/inode-item.h | 16 +
fs/btrfs/inode.c | 592 ++++++++++++++++-----------
fs/btrfs/ioctl.c | 23 +-
fs/btrfs/locking.c | 5 +-
fs/btrfs/lzo.c | 6 +-
fs/btrfs/messages.c | 8 -
fs/btrfs/messages.h | 15 +-
fs/btrfs/misc.h | 20 +
fs/btrfs/ordered-data.c | 442 +++++++++++---------
fs/btrfs/ordered-data.h | 27 +-
fs/btrfs/print-tree.c | 16 +-
fs/btrfs/print-tree.h | 4 +-
fs/btrfs/qgroup.c | 20 +-
fs/btrfs/raid56.c | 49 ++-
fs/btrfs/raid56.h | 3 +
fs/btrfs/relocation.c | 61 +--
fs/btrfs/relocation.h | 3 +-
fs/btrfs/scrub.c | 125 ++----
fs/btrfs/send.c | 16 +-
fs/btrfs/subpage.c | 97 ++---
fs/btrfs/subpage.h | 12 +-
fs/btrfs/super.c | 1 -
fs/btrfs/tests/extent-io-tests.c | 16 +-
fs/btrfs/transaction.c | 16 +-
fs/btrfs/transaction.h | 3 -
fs/btrfs/tree-checker.c | 158 ++++++--
fs/btrfs/tree-checker.h | 29 +-
fs/btrfs/tree-log.c | 62 +--
fs/btrfs/tree-log.h | 2 +-
fs/btrfs/tree-mod-log.c | 257 ++++++++++--
fs/btrfs/volumes.c | 173 ++++----
fs/btrfs/volumes.h | 57 ++-
fs/btrfs/zlib.c | 2 +-
fs/btrfs/zoned.c | 159 ++++----
fs/btrfs/zoned.h | 8 +-
fs/btrfs/zstd.c | 2 +-
include/trace/events/btrfs.h | 39 +-
tools/objtool/check.c | 1 -
75 files changed, 2780 insertions(+), 2710 deletions(-)