qiita icon indicating copy to clipboard operation
qiita copied to clipboard

Download Drops

Open nearinj opened this issue 5 years ago • 10 comments
trafficstars

Hello Qiita devs,

I have been trying to download the BIOM and mapping files for the qiita study 10172, however, everytime my download gets to 37% (8.66Gb) the connection to the server is closed. I have tried on multiple different internet connections and continue to have the same issue.

`index.html?data=biom&study_id 37%[================> ] 8.66G --.-KB/s in 10m 13s

2020-05-19 17:33:32 (14.5 MB/s) - Connection closed at byte 9302100798. Retrying.`

nearinj avatar May 19 '20 20:05 nearinj

In the past we have seen a similar issue when the computer goes to sleep; have you tried: https://qiita.ucsd.edu/static/doc/html/faq.html#how-to-solve-download-or-unzip-errors ?

BTW these might also be helpful:

  • https://qiita.ucsd.edu/static/doc/html/downloading.html.
  • https://qiita.ucsd.edu/static/doc/html/faq.html#do-you-have-specific-qiita-questions

antgonza avatar May 19 '20 20:05 antgonza

I tried all of those options and tested on both a windows pc, and two different PCs running ubuntu 16.04 and have run into the same issue. The file always stops downloading at that byte... I was able to download another study after a few attempts although I also ran into the connection dropping twice before successfully downloading the files.

nearinj avatar May 19 '20 22:05 nearinj

Interesting, which command exactly are you using?

antgonza avatar May 19 '20 22:05 antgonza

On the two different servers:

I have used:

wget 'https://qiita.ucsd.edu/public_download/?data=biom&study_id=10172 '

nearinj avatar May 19 '20 22:05 nearinj

Just a quick note, we are still investigating. It's something specific about one artifact - if you go to https://qiita.ucsd.edu/public/?study_id=10172, you can download all the artifacts except 46193.

cc: @jdereus

antgonza avatar May 23 '20 22:05 antgonza

Awesome thanks for the update and the support. I'll go ahead and use that link for the time being.

nearinj avatar May 24 '20 15:05 nearinj

Hello, I have been downloading the Biom and mapping files of study 13767 and encountered the same problem. It always fails at 76%. Is there any solutions to this? Thank you!

wxgao33 avatar Oct 12 '22 15:10 wxgao33

@wxgao33 Can you share the command you are using?

antgonza avatar Oct 12 '22 15:10 antgonza

wget "https://qiita.ucsd.edu/public_download/?data=biom&study_id=13767" I have tried multiple times and it always fails at 76%(363Gb).

wxgao33 avatar Oct 13 '22 01:10 wxgao33

@wxgao33, thank you for the command, we are investigating.

antgonza avatar Oct 13 '22 13:10 antgonza

A user contacted us via qiita.help letting us know that this was still happening. Then we realized that there was a new mod_zip so we updated both nginx and mod_zip and the user's problems were resolved. Additionally, I just tested the 2 reports here and things seem to also have solved; see notes below.

Anyway, closing for now. If you encounter this error please open another issue or contact qiita.help.

$ wget 'https://qiita.ucsd.edu/public_download/?data=biom&study_id=10172' -O 10172.zip
--2024-01-26 11:57:52--  https://qiita.ucsd.edu/public_download/?data=biom&study_id=10172
Resolving qiita.ucsd.edu (qiita.ucsd.edu)... 10.210.38.19
Connecting to qiita.ucsd.edu (qiita.ucsd.edu)|10.210.38.19|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30194647583 (28G) [application/zip]
Saving to: ‘10172.zip’

100%[======================================================================================================================================================================================>] 30,194,647,583 99.7MB/s   in 6m 24s 

2024-01-26 12:04:57 (75.0 MB/s) - ‘10172.zip’ saved [30194647583/30194647583]

$ wget 'https://qiita.ucsd.edu/public_download/?data=biom&study_id=13767' -O 13767.zip
--2024-01-26 12:05:09--  https://qiita.ucsd.edu/public_download/?data=biom&study_id=13767
Resolving qiita.ucsd.edu (qiita.ucsd.edu)... 10.210.38.19
Connecting to qiita.ucsd.edu (qiita.ucsd.edu)|10.210.38.19|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 271825987329 (253G) [application/zip]
Saving to: ‘13767.zip’

100%[=====================================================================================================================================================================================>] 271,825,987,329 79.6MB/s   in 41m 37s

2024-01-26 12:48:39 (104 MB/s) - ‘13767.zip’ saved [271825987329/271825987329]

antgonza avatar Jan 26 '24 21:01 antgonza