bladebit icon indicating copy to clipboard operation
bladebit copied to clipboard

mega slow Complete writing plot

Open hajes opened this issue 1 year ago • 7 comments

it makes me wonder why renaming/moving plot on same device takes over 60s latest Beta CUDA plotter

/c-tmp/plot-k32-c07-....plot.tmp -> /c-tmp/plot-k32-c07-....plot Completed writing plot in 74.57 seconds

/c-tmp is SSD/NVMe with XFS...after forced trim, it is now 71s.

such a file operation usually takes <1s

Is it a bug?

hajes avatar Aug 13 '23 07:08 hajes

I'm also seeing this behavior.

wallentx avatar Aug 16 '23 06:08 wallentx

Guys from chiaforum said it is normal - explained by Harold himself in Chia Discord.

hajes avatar Aug 16 '23 13:08 hajes

I believe he's making a fix for this.

wallentx avatar Aug 16 '23 15:08 wallentx

Guys from chiaforum said it is normal - explained by Harold himself in Chia Discord.

Completed Plot 1 in 162.94 seconds ( 2.72 minutes )
Completed writing plot in 7.03 seconds

This is currently fixed in a testing branch, and should make its way to dev soon.

wallentx avatar Aug 24 '23 08:08 wallentx

Completed Plot 1 in 151.47 seconds ( 2.52 minutes )

/ramdisk/plot-k32-c07-......plot.tmp -> /ramdisk/plot-k32-c07....plot
Completed writing plot in 18.71 seconds

using ramdisk as suggested by harold

bladebit_cuda 0.0.0-dev I guess latest testing

hajes avatar Aug 26 '23 00:08 hajes

Note that to use tmpfs as a ram output directory on Linux you need to build your own bladebit with this change applied:

--- a/src/plotting/GlobalPlotConfig.h
+++ b/src/plotting/GlobalPlotConfig.h
@@ -33,1 +33,1 @@
-    bool            disableOutputDirectIO  = false;            // Do not use direct I/O when writing the plot files
+    bool            disableOutputDirectIO  = true;             // Do not use direct I/O when writing the plot files

As Linux tmpfs does not support O_DIRECT. It'd be ideal to have that as a command line option, or for PlotWriter() to fall back automatically to non-direct-io when open returns EINVAL.

bladebit can "write" the plot to tmpfs in a seconds this way and you can have a separate process monitoring that to offload it to ssd while bladebit is already working on the next plot.

Also: run ./embed-version.sh before you build if you don't want to see the placeholder 0.0.0-dev version.

gpshead avatar Aug 29 '23 04:08 gpshead

Or use block based ramdisk brd

hajes avatar Aug 29 '23 05:08 hajes