Download dataset speed is very slow

[Sheng1e8]

❓ Questions and Help

Hello thank you for the great framework. When i try to download the dataset, the speed is slower than 2k/s, sometimes even 0b/s. I follow the command as the readme. /# get 1d data python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn /# get 1min data python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data_1min --region cn --interval 1min

Is there some solutions to this? Many thanks.

[you-n-g]

There may be some network issue with our data server.

@SunsetWolf Could you please help to check if https://git-lfs.github.com/ is a better solution for the previous one?

[Smalltarget108]

Same problem.

[SunsetWolf]

Hi! @Sheng1e8 @Smalltarget108 I am sorry to reply you so late, the slow download speed of dataset is because the server is in China, we want to use GLFS tool to move the data to the server outside China, after investigation we found that this tool is chargeable, Qlib has no funding as support, so we can't do this at the moment, if you have a good free big data storage tool, you can tell us. For now, you can try to use VPN to increase the download speed.

[you-n-g]

@SunsetWolf will move the data to AzureBlob to serve the data

[mrtushartiwari]

Can we have direct support using the yahoo Finance API? As it is more faster.

[you-n-g]

It is fixed now. Please try again :)

[github-actions[bot]]

This issue is stale because it has been open for three months with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days