RFE: Bring r/w testing down to Block level.. and add strobing access patterns (memtest'esque)

[benryanau]

Thanks for f3.. OSS RFE's are always hard-asks; so that being said.. F3 could evolve to a new level by bringing its ops down close to bare metal as possible. Current functionality would translate 1:1 for the user.. but with it finally would offer a means to go beyond simple oblique bits in bits back testing.

Much is known of the almost supernatural issues using flash for fs work. USB MSC or SDzz or MMC or straight nandmaps. Linux devs havent stopped tearing their hair out STILL just bughunting consumer-market level tech, nevermind implementing newer stuff (SD phy, MMC/SD command revs, USBxx, UASP, Page size lotto, write amp and erase block sizes, alignment, trim, FTL trickery , cache strategies, device add/remove, bugs and incompatiblilities both device and host side (and at any of the many layers)... filesystem disarray, the arcane tuning in fs params and blkdev settings, flushing,

bitrot disturb loss charge dissipation bricked ftls density^n dismay

chinese binning failed wafers remarking scamming out counterfeits

need i go on? Glad we have USB 3.1 gen 9 and SD PCI-E ThZ and exponentially multiplying and shrinking electrical interfaces.. timing is bad already without adding grime and wear and movement noise.

anyhow sorry. I should have just said RPI, usb keys, smartphone cards and sandforce. Nuff.

If the device was accessed low as possible even blkdev level it would offer so much in allowing comprehensive 100% testing of every layer in all the places solidstate seems to fall down in. And often its subtle and requires specific conditions to uncover failcake uh-ohs. By almost fuzzing access patterns timing bitsets cache and queues hit with exhaustive probing.. all the cogs between driver and flash cell where so much goes mysteriously wrong.

Okay so its beyond fake 128gb usb keys. But what a tool - and the pi ppl might buy you a beer:)

Benchm.. ahem thank you for your consideration. :) Best of luck, we need you!

[AltraMayor]

f3brew already writes and reads at the block level, see make extra.

[SampsonF]

some filesystem like btrfs do not support badblocks.

For now, I can leave those 1GB H2W files that is "corrupted" in place, while removing those "good" ones, so that new data hitting badblocks are less likely.

My Idea is something like this: f3-process /mount_point -> generate a list of "bad" 1GB files f3-process -1GB-filenames /mount_point -> generate a list of "bad" 512KB files f3-process -512MB-filenames /mount_point -> generate a list of "bad" 256KB files ... repeat until reaching 4K (or user can specify a target size)

A final pass is to remove all "good" files so user can use all the "good" spaces.

User need to do filesystem specific preparations:

1. disable COW for the whole filesystem
2. use f3-process to mark "bad" spots of the file system
3. remove "good" files to free up space
4. create a readonly snapshot to freeze those "bad" spot markers
5. create read/write snapshot with COW enabled for normal user