DRAM icon indicating copy to clipboard operation
DRAM copied to clipboard

downloading the databases

Open vdejager opened this issue 1 year ago • 3 comments

Hi,

I want to run DRAMv2 on our HPC, but I'm having trouble downloading data through globus. Since my institute is not in the list, I signed up with my github account, but I can only download data one file at a time. I don't have a desktop machine that will be able the handle the large downloads.

is there an option to directly download the whole dataset as a TAR file to our HPC, where we have plenty of space?

vdejager avatar Nov 29 '24 13:11 vdejager

Just following up on this as another user with challenges in using Globus on our HPC set up as well. Any thoughts to a mirror or a wget compatible URL for the database?

The transfer rate for Globus also seems to be excessively slow from the current host (250 kb/s), so it will take about 3 weeks to download at this rate.

MicroSeq avatar Feb 19 '25 19:02 MicroSeq

Also following up on that - have you found a solution? @vdejager @MicroSeq

utkinaira avatar Mar 30 '25 12:03 utkinaira

Hello, the easiest way to download the whole folder from Globus is probably using globus connect personal (https://docs.globus.org/globus-connect-personal/install/), which allows your laptop, server, etc. to become a globus endpoint. So you can transfer whole folders at once. And it should be faster I believe this way.

madeline-scyphers avatar May 06 '25 18:05 madeline-scyphers