bcachefs
bcachefs copied to clipboard
koverstreet
Bcachefs, self healing reads as "poor man's scrub".
The following untested bash script copies all files of a partition to /dev/null. In the event that bcachefs uses e.g. dupe>=2 or a raid with redundancy, the files are compared with their checksums during the read process by bcachefs' own self-healing and files for which the checksum is not correct are replaced by files for which the checksum of the file is correct.
# [Nihon-Ryori](https://github.com/Nihon-Ryori)
# Open Source Code. Feel free to use the Code for bcachefs project or what ever.
# 2024-11-12, ver 002
# This version should do a Bcachefs, self healing reads as "poor man's scrub" for files, no directory support but now with progress bar
# Untested Code. Use the code at your own risk. Please use the code only if you would write it yourself. The code is only intended for test use with a hard drive of a test system that does not contain any data that is still required.
#!/bin/bash
# Set source and destination directories
SOURCE_DIR="/dev/sdb1" # Replace with actual drive letter/label
DEST_DIR="/dev/null" # Or replace with other drive letter/label
# Initialize counters
total_files=0
processed_files=0
# Check if source and destination drives exist
if ! mountpoint -q "$SOURCE_DIR"; then
echo "Error: Source drive $SOURCE_DIR does not exist."
exit 1
fi
if ! mountpoint -q "$DEST_DIR"; then
echo "Error: Destination drive $DEST_DIR does not exist."
exit 1
fi
# Start copying process
echo "Starting file copy from $SOURCE_DIR to $DEST_DIR"
# Use find to iterate through all files on the source drive
find "$SOURCE_DIR" -type f | while read -r file; do
# Increment counters
((total_files++))
# Copy each file to the destination drive
cp "$file" "$DEST_DIR" || { echo "Error copying $file"; continue; }
# Update processed files counter
((processed_files++))
# Provide progress feedback
echo "Copied: $file"
# Update progress percentage every 100 files
if ((processed_files % 100 == 0)); then
echo "Progress: $(printf "%.1f%%" ((${processed_files} / ${total_files}) * 100)) Complete"
fi
done
echo "poor man's scrub completed."
Untested Code. Use the code at your own risk. Please use the code only if you would write it yourself. The code is only intended for test use with a hard drive of a test system that does not contain any data that is still required.
Open Source Code. Feel free to use the Code for bcachefs project or what ever.
The follow version should check directory's and files and not only files:
# [Nihon-Ryori](https://github.com/Nihon-Ryori)
# Open Source Code. Feel free to use the Code for bcachefs project or what ever.
# 2024-11-12, ver 003
# This version should do a Bcachefs, self healing reads as "poor man's scrub" for files, directorys and now with progress bar
# Untested Code. Use the code at your own risk. Please use the code only if you would write it yourself. The code is only intended for test use with a hard drive of a test system that does not contain any data that is still required.
#!/bin/bash
# Set source and destination directories
SOURCE_DIR="/dev/sdb1" # Replace with actual drive letter/label
DEST_DIR="/dev/null" # Or replace with other drive letter/label
# Initialize counters
total_items=0
processed_items=0
# Check if source directory exists
if ! [ -d "$SOURCE_DIR" ]; then
echo "Error: Source directory $SOURCE_DIR does not exist."
exit 1
fi
# Start copying process
echo "Starting copy from $SOURCE_DIR to $DEST_DIR"
# Use find to iterate through all items on the source directory
find "$SOURCE_DIR" | while read -r item; do
# Increment counters
((total_items++))
# Copy each item to the destination directory
cp -R "$item" "$DEST_DIR" || { echo "Error copying $item"; continue; }
# Update processed items counter
((processed_items++))
# Provide progress feedback
echo "Copied: $item"
# Update progress percentage every 100 items
if ((processed_items % 100 == 0)); then
echo "Progress: $(printf "%.1f%%" ((${processed_items} / ${total_items}) * 100)) Complete"
fi
done
echo "poor man's scrub completed."
The follow version should check directory's and files and not only files:
poor_mans_scrub.sh
# [Nihon-Ryori](https://github.com/Nihon-Ryori)
# Open Source Code. Feel free to use the Code for bcachefs project or what ever.
# 2024-11-15, ver 004
# This version should do a Bcachefs, self healing reads as "poor man's scrub" for files, directory's and now with progress bar, The script close on the end, after pressing enter.
# Untested Code. Use the code at your own risk and only if you would write it yourself. The code is only intended for test use with a test system that does not contain any data that is still required.
#!/bin/bash
# Set source and destination directories
SOURCE_DIR="/dev/sdb1" # Replace with actual drive letter/label
DEST_DIR="/dev/null" # Or replace with other drive letter/label
# Initialize counters
total_items=0
processed_items=0
# Check if source directory exists
if ! [ -d "$SOURCE_DIR" ]; then
echo "Error: Source directory $SOURCE_DIR does not exist."
exit 1
fi
# Start copying process
echo "Starting copy from $SOURCE_DIR to $DEST_DIR"
# Use find to iterate through all items on the source directory
find "$SOURCE_DIR" | while read -r item; do
# Increment counters
((total_items++))
# Copy each item to the destination directory
cp -R "$item" "$DEST_DIR" || { echo "Error copying $item"; continue; }
# Update processed items counter
((processed_items++))
# Provide progress feedback
echo "Copied: $item"
# Update progress percentage every 100 items
if ((processed_items % 100 == 0)); then
echo "Progress: $(printf "%.1f%%" ((${processed_items} / ${total_items}) * 100)) Complete"
fi
done
echo "poor man's scrub completed."
echo "Press Enter to end the script"; read -r
Here's a similar script:
#!/bin/bash
set -euxo pipefail
root=/
sudo time nice ionice -n 7 find "${root}" -mount -type f -print0 \
| shuf --zero-terminated \
| time sudo xargs -0 --max-procs=20 -- ionice -n 7 nice cat \
| nice ionice -n 7 pv --discard
