fairly icon indicating copy to clipboard operation
fairly copied to clipboard

Add the possibility to add a random sleep when storing remote datasets

Open d-consoli opened this issue 1 year ago • 1 comments

Hi, thanks for your work, it is a really useful tool! Unfortunately, when using it to download datasets from Zenodo that contain several (small) files, Zenodo receives too many requests and stops the download. This can be easily solved, for instance, by adding a random sleep in the for loop for downloading the remote datasets, and this is what I did in my fork of your project in this commit. If you think it can be a useful feature and you agree in the way I implemented it I can add an ad-hoc test and create a pull request to your repository, or alternatively search for a better solution. Thanks for your help!

d-consoli avatar Jul 11 '23 12:07 d-consoli