bcachefs
bcachefs copied to clipboard
koverstreet
i_sectors underflow, i_blocks underflow, incorrect i_sectors on some inodes
The error happens reliably on some files: arch/x86/boot/bzImage, and another file that's only ever written to by sponge, a cli tool from moreutils that does something like a write, close, fsync, atomic rename sequence. From last july or earlier, and still happening with current kernels built from bcachefs/master.
From dmesg:
nov. 12 09:14:52 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 205560064 i_sectors underflow: 608 + -616 < 0
nov. 12 09:14:52 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 205560064 i_sectors underflow: 0 + -8 < 0
nov. 12 09:14:52 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 205560064 i_blocks underflow: 29216 + -29232 < 0 (ondisk 29216)
Running bcachefs fsck, I see the same file again appearing repeatedly:
bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): incorrect i_sectors: got 29264, should be 29272
)1321694924.6311252 ta detcennocsid(egamIzb/)1321694924.6311252 ta detcennocsid(egamIzb/toob/)1321694924.4311252 ta detcennocsid(egamIzb/toob/)1321694924.8424052 ta detcen
nocsid(egamIzb/toob/68x/)1321694924.8424052 ta detcennocsid(egamIzb/toob/68x/)1321694924.8182942 ta detcennocsid(egamIzb/toob/68x/hcra/)1321694924.8182942 ta detcennocsid(eg
amIzb/toob/68x/hcra/)1321694924.5959842 ta detcennocsid(egamIzb/toob/68x/hcra/sfehcacb/)1321694924.5959842 ta detcennocsid(egamIzb/toob/68x/hcra/sfehcacb/)1321694924.4959842
ta detcennocsid(egamIzb/toob/68x/hcra/sfehcacb/tig/)1321694924.9640342 ta detcennocsid(egamIzb/toob/68x/hcra/sfehcacb/tig/gro.etaripeiplive/)1321694924.9640342 ta detcennoc
sid(egamIzb/toob/68x/hcra/sfehcacb/tig/gro.etaripeiplive/)1321694924.6904 ta detcennocsid(egamIzb/toob/68x/hcra/sfehcacb/tig/gro.etaripeiplive/crs/)1321694924.6904 ta detcen
nocsid(/src/evilpiepirate.org/git/bcachefs/arch/x86/boot/bzImage, fixing
bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): incorrect i_sectors: got 29216, should be 29232
/src/evilpiepirate.org/git/bcachefs/arch/x86/boot/bzImage, fixing
bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): incorrect i_sectors: got 29200, should be 29216
/src/evilpiepirate.org/git/bcachefs/arch/x86/boot/bzImage, fixing
It seems strange that the error is corrected multiple times for the same inode; is it possible that there are duplicate accounting entries for these problematic inodes?
External UUID: 2c9b10da-e32f-44ab-b303-d2cd1005acf2
Internal UUID: ee332d9f-f329-4f7f-8037-91cea1c3bdda
Magic number: c68573f6-66ce-90a9-d96a-60cf803df7ef
Device index: 0
Label: Home
Version: reconcile (1.33)
Incompatible features allowed: reconcile (1.33)
Incompatible features in use: reconcile (1.33)
Version upgrade complete: reconcile (1.33)
Oldest version on disk: extent_flags (1.25)
Created: Wed Apr 30 09:15:57 2025
Sequence number: 11073
Time of last write: Thu Nov 20 18:35:14 2025
Superblock size: 6.79 KiB/1.00 MiB
Clean: 1
Devices: 2
Sections: crypt,replicas_v0,disk_groups,clean,journal_seq_blacklist,journal_v2,counters,members_v2,errors,ext,downgrade,recovery_passes,extent_type_u64s
Features: lz4,zstd,journal_seq_blacklist_v3,reflink,new_siphash,inline_data,new_extent_overwrite,btree_ptr_v2,extents_above_btree_updates,btree_updates_journalled,reflink_inline_data,new_varint,journal_no_flush,alloc_v2,extents_across_btree_nodes,incompat_version_field,casefolding
Compat features: alloc_info,alloc_metadata,extents_above_btree_updates_done,bformat_overflow_done
Options:
block_size: 4.00 KiB
btree_node_size: 256 KiB
errors: continue [fix_safe] panic ro
write_error_timeout: 30
metadata_replicas: 2
data_replicas: 1
metadata_replicas_required: 1
data_replicas_required: 1
encoded_extent_max: 64.0 KiB
metadata_checksum: none crc32c crc64 [xxhash]
data_checksum: none crc32c crc64 [xxhash]
checksum_err_retry_nr: 3
compression: lz4
background_compression: zstd
str_hash: crc32c crc64 [siphash]
metadata_target: none
foreground_target: ssd
background_target: hdd
promote_target: ssd
erasure_code: 0
casefold: 0
inodes_32bit: 1
shard_inode_numbers_bits: 4
inodes_use_key_cache: 1
gc_reserve_percent: 8
gc_reserve_bytes: 0 B
root_reserve_percent: 0
wide_macs: 0
promote_whole_extents: 1
acl: 1
usrquota: 0
grpquota: 0
prjquota: 0
degraded: [ask] yes very no
journal_flush_delay: 1000
journal_flush_disabled: 0
journal_reclaim_delay: 100
journal_transaction_names: 1
allocator_stuck_timeout: 30
version_upgrade: [compatible] incompatible none
nocow: 0
reconcile_on_ac_only: 0
errors (size 232):
reconcile_bp_to_missing_btree_ptr 89916 Thu Nov 20 18:35:10 2025
btree_ptr_to_bad_reconcile_bp 89807 Thu Nov 20 18:35:10 2025
inode_i_sectors_wrong 4 Thu Nov 20 17:47:46 2025
inode_dir_has_nonzero_i_size 25879639 Wed Nov 19 22:20:52 2025
inode_parent_has_case_insensitive_not_set 5 Wed Nov 19 22:09:28 2025
extent_io_opts_not_set 9 Wed Nov 19 11:48:08 2025
accounting_mismatch 7 Tue Nov 18 17:59:55 2025
accounting_key_underflow 2 Tue Nov 18 17:54:35 2025
vfs_inode_i_blocks_underflow 68 Tue Nov 18 15:14:04 2025
inode_i_sectors_underflow 72 Tue Nov 18 15:14:04 2025
reconcile_work_incorrectly_set 7499 Mon Nov 17 19:05:54 2025
inode_wrong_backpointer 2 Tue Sep 9 08:20:06 2025
vfs_bad_inode_rm 600 Mon Sep 8 08:47:57 2025
lru_entry_bad 1 Fri May 23 08:47:01 2025
ext (size 96):
Recovery passes required:
Errors to silently fix:
Btrees with missing data:
Device 0: /dev/dm-17 (unknown model)
Label: hdd
UUID: 6b61a3e6-529b-4e09-a7c1-84c7670f64b0
Size: 3.64 TiB
read errors: 0
write errors: 0
checksum errors: 0
seqread iops: 0
seqwrite iops: 0
randread iops: 0
randwrite iops: 0
Bucket size: 2.00 MiB
First bucket: 0
Buckets: 1907594
Last mount: Thu Nov 20 10:44:32 2025
Last superblock write: 11073
State: rw
Data allowed: journal,btree,user
Has data: btree,user
Btree allocated bitmap blocksize: 128 MiB
Btree allocated bitmap: 0000000000000000000000000000001100000010000000100000001010010111
Durability: 1
Discard: 0
Freespace initialized: 1
Resize on mount: 0
Last device name: dm-17
Last device model:
Device 1: /dev/dm-16 (unknown model)
Label: ssd
UUID: dff9852b-6fcc-44fb-b0b8-73b278cbffad
Size: 128 GiB
read errors: 0
write errors: 0
checksum errors: 0
seqread iops: 0
seqwrite iops: 0
randread iops: 0
randwrite iops: 0
Bucket size: 2.00 MiB
First bucket: 0
Buckets: 65536
Last mount: Thu Nov 20 10:44:32 2025
Last superblock write: 11073
State: rw
Data allowed: journal,btree,user
Has data: btree,cached
Btree allocated bitmap blocksize: 4.00 MiB
Btree allocated bitmap: 0011111111111111111111111111111111111111111111111111111111111111
Durability: 1
Discard: 1
Freespace initialized: 1
Resize on mount: 0
Last device name: dm-16
Last device model:
Here is some list_journal output, after I got this message in dmesg:
2025-11-26T11:14:47.546105+00:00 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 36894648 i_sectors underflow: 216 + -224 < 0
2025-11-26T11:14:47.546307+00:00 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 36894648 i_blocks underflow: 4200 + -4208 < 0 (ondisk 4200)
bcachefs list_journal -lk inodes:0:36894648 /dev/Consolidated/Home*
https://sebsauvage.net/paste/?2564d1be78542ff7#w7sNvyAHBxuIohuYP/821GQDBSzVE63km3lSRvU3WWE=
Unfortunately it was a few hours after I got the log message, and I've no idea of the size of the journal or of how to parse the timestamps in that log (looking like 18159413314913854). So maybe it doesn't contain the actual incident, just later attempts to produce the warning by rerunning the same commands.
Here is a journal dump 8 minutes after the log message:
https://sebsauvage.net/paste/?902c87fec9c65720#eG8SLjA7pgawxqece6KrWQ0hnR86ObOt2/UkdAGGqFA=
2025-11-28T14:01:00.438102+00:00 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 36894648 i_sectors underflow: 272 + -280 < 0
2025-11-28T14:01:00.456635+00:00 moulinex kernel: bcachefs (2c9b10da-e32f-44ab-b303-d2cd1005acf2): inode 36894648 i_blocks underflow: 4208 + -4216 < 0 (ondisk 4208)