awesome-akash icon indicating copy to clipboard operation
awesome-akash copied to clipboard

Chia - rclone upload all plots at once (parallel upload problem)

Open dzeto opened this issue 2 years ago • 4 comments

Can you please check why parallel uploads on Dropbox through Rclone doesn't work? It's uploading only 1 plot at once, but not all in parallel when they are finished. I tried different options but can't find what is causing a problem.

I use below configuration in manifest:

  • RCLONE=true
  • TOTAL_UPLOADS=8
  • ENDPOINT_LOCATION=dropbox
  • ENDPOINT_DIR="0"
  • JSON_RCLONE=....

dzeto avatar Sep 25 '22 22:09 dzeto

Below rclone comand works on my local Linux servers, but don't know how to add that command on Akash cloud as workaround solution for Chia plotter. Any help would be really appriciated. Thank you!

EXAMPLE: while true; do rclone move /tmp/plots/ --include "*.plot" /mnt/googledrive/ ; sleep 30; done & disown

dzeto avatar Sep 25 '22 23:09 dzeto

Please see : https://docs.akash.network/deploy/chia-on-akash#rclone-to-google-drive You need to put the contents of your rclone.conf into JSON_RCLONE= variable and escape each line with a \n

88plug avatar Sep 26 '22 06:09 88plug

I did like below, but it's only uploading 1 plot per once. On plot management page i see only 1 log file for currently uploading plots. All new plots don't start auto-uploading in parallel, but all waits on each other to finish first. I use crypt option for dropbox so my final distination is: dropbox-crypt. Thank you for help.

        JSON_RCLONE=
        [dropbox]\n
        type = dropbox\n
        chunk_size = 149M\n
        token = {"access_token":"example","token_type":"bearer","refresh_token":"example","expiry":"example"}\n
        [dropbox-crypt]\n
        type = crypt\n 
        remote = dropbox:/backup/\n
        password = example\n
        password2 = example\n
        no_data_encryption = true

chiaaaaaaaaaaaaaaaaaaa

dzeto avatar Sep 28 '22 16:09 dzeto

Can you please try again with the latest release : cryptoandcoffee/akash-chia:316

88plug avatar Dec 08 '22 19:12 88plug