Experiencing download problems

[armbruer]

Hello @jrebut @ArthurOuaknine ,

I'm trying to download the raw dataset from google drive but keep experiencing frequent abortion from upstream.

Tried the following methods:

• gdown a single file
• gdown the whole folder
• manual downloading of single files

The first two methods result in this (tried different files):

Access denied with the following error:

        Too many users have viewed or downloaded this file recently. Please
        try accessing the file again later. If the file you are trying to
        access is particularly large or is shared with many people, it may
        take up to 24 hours to be able to view or download the file. If you
        still can't access a file after 24 hours, contact your domain
        administrator. 

You may still be able to access the file from the browser:

         https://drive.google.com/uc?id=1SXZWMqxwuF5vVoZLgKWhjQYB-FQUR59w 

While the third method of manually downloading randomly aborts most of the time and I have to restart the downloading process (have been able to download at least 3 files).

Any help would be really appreciated.

Thanks, Eric

[HaoranZhuExplorer]

Hi, I also encountered this issue. I spent a lot of time to solve that issue and I think there are only two solutions:

(1) If you want to download the datasets using the web browser, first copy the dataset into your own google drive space and then download all the files using the copied dataset. (Usually a bit slower)

(2) If you want to download the datasets using command line. First, copy the dataset into your google drive space and then try this: https://stackoverflow.com/questions/65312867/how-to-download-large-file-from-google-drive-from-terminal-gdown-doesnt-work. You can get the file_id and use OAuth to get access token, which are necessary to download the files. Finally, using curl to download the files. (much much faster).

[HaoranZhuExplorer]

See my shared google colab notebook (https://colab.research.google.com/drive/1XwvHgbpOGZkrsX0y0LI9EWThZndN8SwB?usp=sharing), it lists the code in which how you first get file_id of each dataset file, then use my previous shared stackoverflow link to get access token to that file_id and finally use curl to download that file.

[coding-and-coding]

See my shared google colab notebook (https://colab.research.google.com/drive/1XwvHgbpOGZkrsX0y0LI9EWThZndN8SwB?usp=sharing), it lists the code in which how you first get file_id of each dataset file, then use my previous shared stackoverflow link to get access token to that file_id and finally use curl to download that file.

hi, the raw data is not available in google drive now, can you share it with me via some ways? thanks!

[HaoranZhuExplorer]

See my shared google colab notebook (https://colab.research.google.com/drive/1XwvHgbpOGZkrsX0y0LI9EWThZndN8SwB?usp=sharing), it lists the code in which how you first get file_id of each dataset file, then use my previous shared stackoverflow link to get access token to that file_id and finally use curl to download that file.

hi, the raw data is not available in google drive now, can you share it with me via some ways? thanks!

Sorry, we previously decided to not proceed the experiments on this dataset so I deleted the data on my machine. You may contact the author for the data.

[coding-and-coding]

See my shared google colab notebook (https://colab.research.google.com/drive/1XwvHgbpOGZkrsX0y0LI9EWThZndN8SwB?usp=sharing), it lists the code in which how you first get file_id of each dataset file, then use my previous shared stackoverflow link to get access token to that file_id and finally use curl to download that file.

hi, the raw data is not available in google drive now, can you share it with me via some ways? thanks!

Sorry, we previously decided to not proceed the experiments on this dataset so I deleted the data on my machine. You may contact the author for the data.

thanks!