| Age | Commit message (Collapse) | Author |
|---|
|
This wasn't updated when we added tracking for btree validate errors.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Add a tracepoint for any time we return an error and unwind.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
The new guard(), scoped_guard() allow for more natural code.
Some of the uses with creative flow control have been left.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We'd like users to be able to debug without building custom kernels, so
this will help us get rid of CONFIG_BCACHEFS_DEBUG, at least for most
things.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Preferentially keep existing cached pointers instead of adding new ones.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Pretty printer for bch_io_failures, to be used for better read error
messages.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Part of the ongoing project to kill off bch2_(fs|trans)_inconsistent
calls - they generally need to be replaced with either
- a fsck_err() call that can repair the error, or
- logging an error of the appropriate type in the superblock, and
flagging the appropriate recovery pass to repair the error
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Now, if an extent is poisoned we can move it even if there was a
checksum error. We'll have to give it a new checksum, but the poison bit
means that userspace will still see the appropriate error when they try
to read it.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
If promote target isn't set, rebalance should still leave a cached copy
on the faster device.
Fall back to foreground_target if it's set, or allow a cached copy on
any device if neither are set.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
extent poisoning is partly so that we don't keep spewing the dmesg log
when we've got unreadable data - we don't want to print these.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
syzbot discovered that this one is possible: we have pointers, but none
of them are to valid devices.
Reported-by: syzbot+336a6e6a2dbb7d4dba9a@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We store to all fields, so the kmsan warnings were spurious - but
initializing via stores to bitfields appear to have been giving the
compiler/kmsan trouble, and they're not necessary.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
It's possible for checksum errors to be transient - e.g. flakey
controller or cable, thus we need additional retries (besides retrying
from different replicas) before we can definitely return an error.
This is particularly important for the next patch, which will allow the
data move path to move extents with checksum errors - we don't want to
accidentally introduce bitrot due to a transient error!
- bch2_bkey_pick_read_device() is substantially reworked, and
bch2_dev_io_failures is expanded to record more information about the
type of failure (i.e. number of checksum errors).
It now returns an error code that describes more precisely the reason
for the failure - checksum error, io error, or offline device, instead
of the previous generic "insufficient devices". This is important for
the next patches that add poisoning, as we only want to poison extents
when we've got real checksum errors (or perhaps IO errors?) - not
because a device was offline.
- Add a new option and superblock field for the number of checksum
retries.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Previously, we woudn't try to read at all from a failed device - that
doesn't make much sense, the device may be unhealthy (perhaps taking
longer than it should to service reads), but if it's our only option we
should still try to read from it.
Now, bch2_bkey_pick_read_device() will pick failed devices only if there
are no non-failed replicas to read from.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This implements a new extent field bitflags that apply to the whole
extent. There's been a couple things we've wanted this for in the past,
but the immediate need is extent poisoning, to solve a rebalance issue.
Unknown extent fields can't be parsed (we won't known their size, so we
can't advance to the next field), so this is an incompat feature, and
using it prevents the filesystem from being mounted by old versions.
This also adds the BCH_EXTENT_poisoned flag; this indicates that the
data is known to be bad (i.e. there was a checksum error, and we had to
write a new checksum) and reads will return errors.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
To be used for scrub, where we want the read to come from a specific
device.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
steal the (clever) algorithm from get_random_u32_below()
this fixes a bug where we were passing roundup_pow_of_two() a 64 bit
number - we're squaring device latencies now:
[ +1.681698] ------------[ cut here ]------------
[ +0.000010] UBSAN: shift-out-of-bounds in ./include/linux/log2.h:57:13
[ +0.000011] shift exponent 64 is too large for 64-bit type 'long unsigned int'
[ +0.000011] CPU: 1 UID: 0 PID: 196 Comm: kworker/u32:13 Not tainted 6.14.0-rc6-dave+ #10
[ +0.000012] Hardware name: ASUS System Product Name/PRIME B460I-PLUS, BIOS 1301 07/13/2021
[ +0.000005] Workqueue: events_unbound __bch2_read_endio [bcachefs]
[ +0.000354] Call Trace:
[ +0.000005] <TASK>
[ +0.000007] dump_stack_lvl+0x5d/0x80
[ +0.000018] ubsan_epilogue+0x5/0x30
[ +0.000008] __ubsan_handle_shift_out_of_bounds.cold+0x61/0xe6
[ +0.000011] bch2_rand_range.cold+0x17/0x20 [bcachefs]
[ +0.000231] bch2_bkey_pick_read_device+0x547/0x920 [bcachefs]
[ +0.000229] __bch2_read_extent+0x1e4/0x18e0 [bcachefs]
[ +0.000241] ? bch2_btree_iter_peek_slot+0x3df/0x800 [bcachefs]
[ +0.000180] ? bch2_read_retry_nodecode+0x270/0x330 [bcachefs]
[ +0.000230] bch2_read_retry_nodecode+0x270/0x330 [bcachefs]
[ +0.000230] bch2_rbio_retry+0x1fa/0x600 [bcachefs]
[ +0.000224] ? bch2_printbuf_make_room+0x71/0xb0 [bcachefs]
[ +0.000243] ? bch2_read_csum_err+0x4a4/0x610 [bcachefs]
[ +0.000278] bch2_read_csum_err+0x4a4/0x610 [bcachefs]
[ +0.000227] ? __bch2_read_endio+0x58b/0x870 [bcachefs]
[ +0.000220] __bch2_read_endio+0x58b/0x870 [bcachefs]
[ +0.000268] ? try_to_wake_up+0x31c/0x7f0
[ +0.000011] ? process_one_work+0x176/0x330
[ +0.000008] process_one_work+0x176/0x330
[ +0.000008] worker_thread+0x252/0x390
[ +0.000008] ? __pfx_worker_thread+0x10/0x10
[ +0.000006] kthread+0xec/0x230
[ +0.000011] ? __pfx_kthread+0x10/0x10
[ +0.000009] ret_from_fork+0x31/0x50
[ +0.000009] ? __pfx_kthread+0x10/0x10
[ +0.000008] ret_from_fork_asm+0x1a/0x30
[ +0.000012] </TASK>
[ +0.000046] ---[ end trace ]---
Reported-by: Roland Vet <vet.roland@protonmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This lets us print the exact location in the journal if it was found in
the journal, or correctly print if it was found in the superblock.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
When not compressed, these must be equal - this fixes an assertion pop
in bch2_rechecksum_bio().
Reported-by: syzbot+50d3544c9b8db9c99fd2@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Add a new parameter to bkey validate functions, and use it to improve
invalid bkey error messages: we can now print the btree and depth it
came from, or if it came from the journal, or is a btree root.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Per reports of performance issues on mixed multi device filesystems
where we're issuing too much IO to the spinning rust - tweak this
algorithm.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Better repair for reflink pointers, as well as propagating new inode
options to indirect extents, are going to require a few extra bits
bch_reflink_p: so claim a few from the high end of the destination
index.
Also add some missing bounds checking.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We now say explicitly which pointers are being moved or compressed
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Since bch2_move_get_io_opts() now synchronizes io_opts with options from
bch_extent_rebalance, delete the ad-hoc logic in rebalance.c that
previously did this.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
bch2_move_get_io_opts() now synchronizes options loaded from the
filesystem and inode (if present, i.e. not walking the reflink btree
directly) with options from the bch_extent_rebalance_entry, updating the
extent if necessary.
Since bch_extent_rebalance tracks where its option came from we can
preserve "inode options override filesystem options", even for indirect
extents where we don't have access to the inode the options came from.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
- Add more io path options to bch_extent_rebalance
- For each option, track whether it came from the filesystem or the
inode
This will be used for improved rebalance support for reflinked data.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
New helper to simplify bch2_bkey_set_needs_rebalance()
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Single caller, fold into bch2_bkey_sectors_need_rebalance()
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Dead code
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Centralize some io path option fixups - they weren't always being
applied correctly:
- background_compression uses compression if unset
- background_target uses foreground_target if unset
- nocow disables most fancy io path options
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
A user popped up with a very old (0.11) filesystem that needed repair
and wasn't recently backed up.
Reported-by: Manoa <manoa@mail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This runs on extents that haven't yet been validated, so we don't want
to assert that we have a valid entry type.
Reported-by: syzbot+4f29c3f12f864d8a8d17@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We had a bug report where the data update path was creating an extent
that failed to validate because it had too many pointers; almost all of
them were cached.
To fix this, we have:
- want_cached_ptr(), a new helper that checks if we even want a cached
pointer (is on appropriate target, device is readable).
- bch2_extent_set_ptr_cached() now only sets a pointer cached if we want
it.
- bch2_extent_normalize_by_opts() now ensures that we only have a single
cached pointer that we want.
While working on this, it was noticed that this doesn't work well with
reflinked data and per-file options. Another patch series is coming that
plumbs through additional io path options through bch_extent_rebalance,
with improved option handling.
Reported-by: Reed Riley <reed@riley.engineer>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We want to be using private errcodes whenever possible, for better error
messages.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
bch2_dev_rcu() now properly errors if the device is invalid
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
bch2_bkey_drop_ptrs() had a some complicated machinery for avoiding
O(n^2) when dropping multiple pointers - but when n is only going to be
~4, it's not worth it.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This was caught as a very rare nonce inconsistency, on systems with
encryption and replication (and tiering, or some form of rebalance
operation running):
[Wed Jul 17 13:30:03 2024] about to insert invalid key in data update path
[Wed Jul 17 13:30:03 2024] old: u64s 10 type extent 671283510:6392:U32_MAX len 16 ver 106595503: durability: 2 crc: c_size 8 size 16 offset 0 nonce 0 csum chacha20_poly1305_80 compress zstd ptr: 3:355968:104 gen 7 ptr: 4:513244:48 gen 6 rebalance: target hdd compression zstd
[Wed Jul 17 13:30:03 2024] k: u64s 10 type extent 671283510:6400:U32_MAX len 16 ver 106595508: durability: 2 crc: c_size 8 size 16 offset 0 nonce 0 csum chacha20_poly1305_80 compress zstd ptr: 3:355968:112 gen 7 ptr: 4:513244:56 gen 6 rebalance: target hdd compression zstd
[Wed Jul 17 13:30:03 2024] new: u64s 14 type extent 671283510:6392:U32_MAX len 8 ver 106595508: durability: 2 crc: c_size 8 size 16 offset 0 nonce 0 csum chacha20_poly1305_80 compress zstd ptr: 3:355968:112 gen 7 cached ptr: 4:513244:56 gen 6 cached rebalance: target hdd compression zstd crc: c_size 8 size 16 offset 8 nonce 0 csum chacha20_poly1305_80 compress zstd ptr: 1:10860085:32 gen 0 ptr: 0:17285918:408 gen 0
[Wed Jul 17 13:30:03 2024] bcachefs (cca5bc65-fe77-409d-a9fa-465a6e7f4eae): fatal error - emergency read only
bch2_extents_match() was reporting true for extents that did not
actually point to the same data.
bch2_extent_match() iterates over pairs of pointers, looking for
pointers that point to the same location on disk (with matching
generation numbers). However one or both extents may have been trimmed
(or merged) and they might not have the same disk offset: it corrects
for this by subtracting the key offset and the checksum entry offset.
However, this failed when an extent was immediately partially
overwritten, and the new overwrite was allocated the next adjacent disk
space.
Normally, with compression off, this would never cause a bug, since the
new extent would have to be immediately after the old extent for the
pointer offsets to match, and the rebalance index update path is not
looking for an extent outside the range of the extent it moved.
However with compression enabled, extents take up less space on disk
than they do in the btree index space - and spuriously matching after
partial overwrite is possible.
To fix this, add a secondary check, that strictly checks that the
regions pointed to on disk overlap.
https://github.com/koverstreet/bcachefs/issues/717
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
rebalance_work was keying off of the presence of rebelance_opts in the
extent - but that was incorrect, we keep those around after rebalance
for indirect extents since the inode's options are not directly
available
Fixes: 20ac515a9cc7 ("bcachefs: bch_acct_rebalance_work")
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We don't have sufficient information to debug:
https://github.com/koverstreet/bcachefs/issues/726
- print out durability of extent ptrs, when non default
- print the number of replicas we need in data_update_to_text()
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
bkey_fsck_err() was added as an interface that looks like fsck_err(),
but previously all it did was ensure that the appropriate error counter
was incremented in the superblock.
This is a cleanup and bugfix patch that converts it to a wrapper around
fsck_err(). This is needed to fix an issue with the upgrade path to
disk_accounting_v3, where the "silent fix" error list now includes
bkey_fsck errors; fsck_err() handles this in a unified way, and since we
need to change printing of bkey fsck errors from the caller to the inner
bkey_fsck_err() calls, this ends up being a pretty big change.
Als,, rename .invalid() methods to .validate(), for clarity, while we're
changing the function signature anyways (to drop the printbuf argument).
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This repurposes the promote path, which already knows how to call
data_update() after a read: we now automatically rewrite bad data when
we get a read error and then successfully retry from a different
replica.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Turn more asserts into proper recoverable error paths.
Reported-by: syzbot+246b47da27f8e7e7d6fb@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We're about to start using bch_validate_flags for superblock section
validation - it's no longer bkey specific.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
