bladebit icon indicating copy to clipboard operation
bladebit copied to clipboard

fallocate plot files in background upon creation

Open mocksoul opened this issue 3 years ago • 1 comments

Once hdd is being filled up to 80%+ allocating 101.5GiB of data can take up to 30-60 extra seconds. That is usually done during plot writing stage (3, 4 and waiting for prev plot in stage 1) by filesystem itself while plot file is being written.

In this patch i'm using fallocate system call and run it in background thread after plot file creation. Linux optimizes this -- fallocate process take lowest IO prio in kernel -- so, it does not affect previous plot writing speeds at all.

This can have different impact on various filesystems, I'm using ext4 with data=writeback.

Same can be archieved by separate fallocate process on zero sized *.plot.tmp files, e.g: while true; do find /mnt/plots_mountpoint -name '*.plot.tmp' -size 0 -print -exec fallocate -l 101G {} \;; sleep 60; done

Example bladebit output:

Generating F1...
Fallocating files...
Finished F1 generation in 2.64 seconds.
Sorting F1...
Fallocate succeeded in 15.47 secs
Finished F1 sort in 43.67 seconds.

This is newly created filesystem on 18TB disk. On 80%+ filled disks fallocate syscall takes ~1min and happens in separate thread.

mocksoul avatar Aug 01 '21 21:08 mocksoul

I'm fallocate-ing 101GiB, because avg plot size is around 101.35GiB and smallest I have seen is 101.25GiB (10k+ plots). It could be possible to allocate more here (e.g. 102GiB), but that will require ftruncate after all writes to plot file done.

But 101GiB works fine for me, so I didnt bothered =).

mocksoul avatar Aug 01 '21 22:08 mocksoul