mdcct icon indicating copy to clipboard operation
mdcct copied to clipboard

Plot Slowdown

Open vadirthedark opened this issue 8 years ago • 24 comments

I am using the mdcct plotter and after hitting the 95% complete level the program has slowed to a crawl or has stopped altogether. Any ideas as to why this might happen. I am plotting a 5TB drive. Here is the input to start:

./plot -k xxxxxxxxxxxxx587 -x 1 -d /media/imbota/burst/ -s 0

Working output:

Using SSE2 core. Creating plots for nonces 0 to 18907000 (4959 GB) using 12775 MB memory and 4 threads 95 percent done.

Thanks for any help you can give.

vadirthedark avatar Nov 14 '16 14:11 vadirthedark

Hmmm.... really can't tell why that would happen. Worked fine for me also for quite pig plots. Which part seems to slow down? Hmmm.... really can't tell why that would happen. Worked fine for me also for quite pig plots. Which part seems to slow down? Plot creating, or writing to HDD?

Mirkic7 avatar Nov 15 '16 08:11 Mirkic7

mine is stuck at 95% too. command was: ./plotavx2 -k numerickey -x 1 -d /path/to/hdd -m 8192 -t 8 -a output is: Async mode set. Using SSE2 core. Adjusting total nonces to 7495680 to match stagger size Creating plots for nonces 427645713711400004 to 427645713718895684 (1966 GB) using 4096 MB memory and 8 threads 95 Percent done. (ASYNC write)

thanks

update: It's still using cpu as if it's working on something. I'm on day two now of 95%.

ITwrx avatar May 24 '17 19:05 ITwrx

Please specify when running start nonce (even if 0), and size nonce's (-s and -n options).

Mirkic7 avatar May 26 '17 06:05 Mirkic7

the reason i didn't include those before is because i don't know what the values should be. i'm guessing i can use 0 for start nonce but i have no idea what to use for the -n option. please advise. thanks

update: i noticed "Adjusting total nonces to 7495680 to match stagger size" from previous output so i'm running "cpulimit -l 50 -i ./plotavx2 -k numericburstid -x 1 -d path/to/hdd -s 0 -n 7495680 -m 8192 -t 8 -a". i'll let you know what happens.

ITwrx avatar May 26 '17 13:05 ITwrx

hi,

I think I have a similar issue. Since 7 hours or so the plotter is at 100% Percent done. (ASYNC Write). No nonces/minute prompt. Two threads are still running (maybe because of async mode). The HDD ist 100% full (df says all Available: 0). The only option I missed is "-n", because I read it can self determine. Should I kill it or wait?

@ITwrx https://github.com/robertbiswas/mjminer-Plot-Command-Generator there you will find a command generator as html. It will translate you GBs in nonces.

BobbyT avatar Jun 11 '17 10:06 BobbyT

@BobbyT thanks for the link. i may end up needing it still. the command i ran 16 days ago is only 55% done due to using cpulimit. i still think that's pretty crazy though. either max out your cpu for days or wait a month to run it at 50%. wow.

ITwrx avatar Jun 11 '17 13:06 ITwrx

@ITwrx you are using ./plotavx2 -k numericburstid -x 1 . Is this by mistake or doesn't your CPU support AVX2? If it does, you should have used -x 2. If I'm reading the plot.c code correctly, AVX2 compared to SSE4 is using double the amount of generating nonces.

btw: I killed the process, started the optimizer and the result was something like this: File not complete. Expected size: , actual size:

The auto determination of nonces doesn't work properly and ends in an deadlock, too bad.

BobbyT avatar Jun 11 '17 17:06 BobbyT

fx8350. shows avx only. i just restarted. would have taken forever to finaish anyways. upped the cpulimit to 80%. going to move to i5 machine that does have avx2.

ITwrx avatar Jun 11 '17 17:06 ITwrx

The -t parameter is for threads and therefore cores. -x is for CPU instruction set, meaning how much instructions a cpu is able to execute.

1 CPU with 4 Cores and something like HyperThreading results in 8 Threads -t 8

btw: the command generator doesn't add ASYNC -a flag

BobbyT avatar Jun 11 '17 17:06 BobbyT

i ran "./plotavx2 -k numericburstid -x 2 -d /mnt -s 0 -n 7495680 -m 2048 -t 4 -a" on an i5 machine with 2 tb hdd.

it's stuck at 95%

ITwrx avatar Jun 12 '17 13:06 ITwrx

that was quick...

what does df tells you, is there still space left on that drive?

BobbyT avatar Jun 12 '17 19:06 BobbyT

An alternative: https://forums.burst-team.us/topic/5998/mjminer-plotter-fork-for-optimized-plots-on-macos-cpu-avx2-sse4 More Detailed: https://steemit.com/burst/@k06a/efficient-burst-mining-on-macos-linux

I will test it tomorrow...

BobbyT avatar Jun 12 '17 21:06 BobbyT

yeah it finished it like 8-12 hrs or something with an i5 and avx2. it still had a little space left. i wonder if my -n value was larger than the space on the drive so it gets hung up trying to continue on. i have 1.8 tb reported by lsblk and my -n value was 7495680. i don't understand how to calculate the -n option even though i read the explanation in the steem article. it seems needlessly convoluted. maybe it's just me...

a "use whole directory/device" option would be nice.

ITwrx avatar Jun 12 '17 22:06 ITwrx

The command generator gives me -n 7372800 for 1800 GB

BobbyT avatar Jun 12 '17 23:06 BobbyT

oh, i thought you were saying that the command generator hadn't worked after all so i never tried it. thanks. i'll try that nonce and see what happens.

ITwrx avatar Jun 12 '17 23:06 ITwrx

there seems to be a discepency in nonce calculation between the command generator, stagger size and plotavx2, as when i used 7372800 plotavx2 calculated the disk size as slightly above 1.8TB and indeed plotavx2 got stuck at 96%. i had to lower nonce value to 7000000 which plotavx2 then showed as 7000064 (1836 GB). that is 1.79 TB and is running now. maybe this time it can complete.

ITwrx avatar Jun 13 '17 13:06 ITwrx

Today I finished plotting and then I wanted to start the optimizer, but I get the error message Not enough free space on device. Free x MB to continue The drive is 99% full and the optimizer wants me to free the whole drive...

any idea what to do?

BobbyT avatar Jun 13 '17 19:06 BobbyT

if you're talking about mjminer it's readme and announcement seem to indicate it plots and optimizes at the same time, so my guess is it wants to replot your whole drive. don't ask me why it has miner in the name with no mining instructions in the readme. i haven't dug any deeper than that though. i'm still currently plotting with mdcct's plotavx2.

ITwrx avatar Jun 13 '17 19:06 ITwrx

no I meant mdcct's plot & optimize.

BobbyT avatar Jun 13 '17 19:06 BobbyT

oh, ok. "./optimize --help" returns nothing. my guess would be to let the optimizer do what it wants (free the space it wants to free) or to mine without running the optimizer. I would hope if you free the space, it optimizes without actually destroying your plot. otherwise, why would we even need to run the plotter to begin with? of course it could be some idiosyncrisity with how you're supposed to call "optimize" but without any help info you just have to guess or wait on the author to respond. did you just run "./optimize"? or something else?

ITwrx avatar Jun 13 '17 20:06 ITwrx

okay, the optimizer needs the same space as the plot itself. It can't optimize in place.

BobbyT avatar Jun 13 '17 22:06 BobbyT

do you mean the optimizer needs the identical amount of free space as the plot (i'm assuming on the same device)? if so, how are you ever supposed to get the last plot optimized?

ITwrx avatar Jun 13 '17 22:06 ITwrx

yes it seems so. There is a GUI version for windows, there you are able to set the output directory for the optimized plots.

BobbyT avatar Jun 13 '17 23:06 BobbyT

thanks for the info. If i can't get Burst mining working on GNU+Linux using FLOSS without too much nonsense, then i just won't be able to mine or buy Burst.

btw, mine finished plotting so the problem was that the nonce value was too high for the drive space i actually had available.

ITwrx avatar Jun 14 '17 00:06 ITwrx