CUDA - Regular plot fails plot check

[Jsewill]

Running chia plots check on uncompressed plots created with bladebit_cuda cudaplot, results in the following error:

2023-02-13T06:59:19.004 chia.plotting.check_plots : ERROR <class 'RuntimeError'>: Src size is incorrect error in getting challenge qualities for plot <plot_filepath>

[XCHSystems]

I have created thousands of plots with bladebit_cuda and they all pass the plot check

[Jsewill]

@FlexiMiners Excellent. I would like to have the same experience.

[XCHSystems]

@Jsewill Tell us more about your system, what GPU, CPU, RAM etc, where you are storing your plots temporarily, where your final destination is. For example in my system

Xeon 6314U 512GB RAM NVIDIA A4000 Plot destination for plotter is 4xP5800X RAID Disk plow.py then moves the plots to spinning disks

Simon

[HomeLH]

@Jsewill Tell us more about your system, what GPU, CPU, RAM etc, where you are storing your plots temporarily, where your final destination is. For example in my system

Xeon 6314U 512GB RAM NVIDIA A4000 Plot destination for plotter is 4xP5800X RAID Disk plow.py then moves the plots to spinning disks

Simon

How to create compressed plot? I can not find compression argument.

[XCHSystems]

@Jsewill Tell us more about your system, what GPU, CPU, RAM etc, where you are storing your plots temporarily, where your final destination is. For example in my system Xeon 6314U 512GB RAM NVIDIA A4000 Plot destination for plotter is 4xP5800X RAID Disk plow.py then moves the plots to spinning disks Simon

How to create compressed plot? I can not find compression argument.

AFAIK Compressed plots are not supported at this moment, that is in one of the next phases of the cuda plotter

[Jsewill]

I have no idea if this is a bug with bladebit_cuda, or something on my end, but I figure it is better that we find out for sure.

Dell R720 2 E5-2690 v2 CPUs 640GB of LRDIMMs RTX 3060

I plotted using an XFS formatted brd RAM disk for the immediate output, then moved using madmax' chia-plot-copy/sink.

After I scanned all of the plots I had made with bladebit_cuda, I found that not all of the plots produce that error. Just over half of them. I tried creating a couple plots since, and will try again soon. I have since tried creating two plots: one directly to an HDD housed in a QSFP connected JBOD, and one in RAM as I had before. Both checked out fine.

We had some electricity quality issues recently, and I wonder if that may have impacted these plots. Those issues have stopped, so I am planning to try a plotting and testing run again.

[XCHSystems]

I have no idea if this is a bug with bladebit_cuda, or something on my end, but I figure it is better that we find out for sure.

Dell R720 2 E5-2690 v2 CPUs 640GB of LRDIMMs RTX 3060

I plotted using an XFS formatted brd RAM disk for the immediate output, then moved using madmax' chia-plot-copy/sink.

After I scanned all of the plots I had made with bladebit_cuda, I found that not all of the plots produce that error. Just over half of them. I tried creating a couple plots since, and will try again soon. I have since tried creating two plots: one directly to an HDD housed in a QSFP connected JBOD, and one in RAM as I had before. Both checked out fine.

We had some electricity quality issues recently, and I wonder if that may have impacted these plots. Those issues have stopped, so I am planning to try a plotting and testing run again.

I have done over 2.5k plots since Saturday and ran a plot check against them all, not one has failed the plot check. Did you check the file size of the affected ones, did they look "Normal"? I am using plow.py to move my plots from the temporary storage to the destination disks (15 disks at a time), it uses RSYNC to perform the copy operation, not sure what plot_sink utilises as I never looked at it. Could be a problem with the copy process.

[Jsewill]

I have no idea if this is a bug with bladebit_cuda, or something on my end, but I figure it is better that we find out for sure. Dell R720 2 E5-2690 v2 CPUs 640GB of LRDIMMs RTX 3060 I plotted using an XFS formatted brd RAM disk for the immediate output, then moved using madmax' chia-plot-copy/sink. After I scanned all of the plots I had made with bladebit_cuda, I found that not all of the plots produce that error. Just over half of them. I tried creating a couple plots since, and will try again soon. I have since tried creating two plots: one directly to an HDD housed in a QSFP connected JBOD, and one in RAM as I had before. Both checked out fine. We had some electricity quality issues recently, and I wonder if that may have impacted these plots. Those issues have stopped, so I am planning to try a plotting and testing run again.

I have done over 2.5k plots since Saturday and ran a plot check against them all, not one has failed the plot check. Did you check the file size of the affected ones, did they look "Normal"? I am using plow.py to move my plots from the temporary storage to the destination disks (15 disks at a time), it uses RSYNC to perform the copy operation, not sure what plot_sink utilises as I never looked at it. Could be a problem with the copy process.

I will move them with a bash script that uses regular mv, to eliminate any problems which may exist in the movement of plots.

[XCHSystems]

I plotted using an XFS formatted brd RAM disk for the immediate output, then moved using madmax' chia-plot-copy/sink

I would typically not format a RAMDISK in Linux, just mount the RAMDISK and use it native, not sure if that is your problem as when I used to use RAMDISK in the past for plotting, I would not create a file system on it, just mount it using fstab

[XCHSystems]

I will move them with a bash script that uses regular mv, to eliminate any problems which may exist in the movement of plots.

using mv does not give you any failback option, using rsync does, as rsync will do a compare before removing the old file

rsync --preallocate --remove-source-files --skip-compress=plot --whole-file -avP <sourcefile> <destination folder>

[Jsewill]

I plotted using an XFS formatted brd RAM disk for the immediate output, then moved using madmax' chia-plot-copy/sink

I would typically not format a RAMDISK in Linux, just mount the RAMDISK and use it native, not sure if that is your problem as when I used to use RAMDISK in the past for plotting, I would not create a file system on it, just mount it using fstab

bladebit refused to plot to a tmpfs RAMDISK, so I had to use brd. Maybe I only thought I tried it again with bladebit_cuda, so I will try that as well.

I will move them with a bash script that uses regular mv, to eliminate any problems which may exist in the movement of plots.

using mv does not give you any failback option, using rsync does, as rsync will do a compare before removing the old file

Indeed, I will use rsync for these tests. I stopped using rsync because it had unnecessary overhead (read took much longer), and I've moved several thousands of plots without a problem, using mv or pv, and even some via scp--but I do see the wisdom in your statement about rsync.

[HomeLH]

@Jsewill Tell us more about your system, what GPU, CPU, RAM etc, where you are storing your plots temporarily, where your final destination is. For example in my system Xeon 6314U 512GB RAM NVIDIA A4000 Plot destination for plotter is 4xP5800X RAID Disk plow.py then moves the plots to spinning disks Simon

How to create compressed plot? I can not find compression argument.

AFAIK Compressed plots are not supported at this moment, that is in one of the next phases of the cuda plotter

I use --compress 7 parameter, it lools like working fine，but chia plots check command can not recognize them.

[XCHSystems]

Compression is not part of the Alpha release, there was an official annoucement that went like this:

Phase 1: ⚙️ GPU Plotting (now) test the new Bladebit cuda plotter with current plot format. You can farm these plots with the current Chia client immediately today. This is where we are right now, no compression!

Phase 2: ⬇️ Compression (3-5 weeks) test compressed plots and simulated farming so that farmers can understand the compute and power overhead for their actual systems using compressed plots. We will also release version of Bladebit ramplot and diskplot that support compression

Phase 3: 🚜 Farming (4-6 weeks) real farming and harvesting of compressed plots

Phase 4 - TBD farming and plotting roadmap priority being discussed for GPU farming, AMD support, higher K values, GUI enhancements, and more

[Smith-Jessica]

I am seeing this same error with a majority of my plots, compressed and uncompressed: chia.plotting.check_plots : ERROR <class 'RuntimeError'>: Src size is incorrect error in getting challenge qualities for plot <plot_filepath>

Here are my specs: Unraid on Dell R730 GTX 1070 Xeon E5-2660v3 320GB DDR4 ECC tmp drive: Samsung PM 983 x2 in Raid 0 I use plotman archiver to move them to hdd, I don't believe this is the issue though as I've run a manual transfer test with rsync and it still showed as invalid with chia plots check