PGPortfolio icon indicating copy to clipboard operation
PGPortfolio copied to clipboard

failed to download data or train

Open franklili3 opened this issue 5 years ago • 13 comments

When I download data or train without downloading data, there is a error " File "/home/ubuntu/PGPortfolio/pgportfolio/marketdata/globaldatamatrix.py", line 199, in __fill_data if c["date"] > 0:TypeError: string indices must be integers". What can I do?

franklili3 avatar Feb 26 '19 03:02 franklili3

I have get the trade paires and volumes." pair price volume coin
ETH BTC_ETH 0.035572 833820.402758 LTC BTC_LTC 0.011680 669757.028884 XRP BTC_XRP 0.000082 465788.871570 reversed_USDT USDT_BTC 0.000263 200241.369072 ETC BTC_ETC 0.001111 165836.536867 DASH BTC_DASH 0.021320 145825.768286 XMR BTC_XMR 0.012554 82908.949198 XEM BTC_XEM 0.000011 69047.989435 FCT BTC_FCT 0.001690 63216.478905"

franklili3 avatar Feb 27 '19 07:02 franklili3

I have the same problem pgportfolio\marketdata\globaldatamatrix.py", line 199, in __fill_data if c["date"] > 0: TypeError: string indices must be integers

hn2 avatar Mar 04 '19 11:03 hn2

I made it work by cloning the dev repo, change the exchange in config file to binance, changing the dates in base config and train_package NNAgent config to a shorter frame such as 2019/1 -> 2019/3

ghost avatar Mar 24 '19 16:03 ghost

This is the reason and solution: {'error': 'Data requested is too large. Please specify a longer period or a shorter date range, or use resolution=auto to automatically calculate the shortest supported period for your date range.'}

JayWu7 avatar Mar 25 '19 09:03 JayWu7

Same here. How to set resolution to auto? No resolution field in net_config.json. In https://github.com/ZhengyaoJiang/PGPortfolio/blob/master/user_guide.md, author mentioned that

Note that using the this file, you shouldn't make any changes to input data configuration(For example start_date, end_date or coin_number) otherwise incorrect result might be presented.

but looks like I can only shorten the start_date and end_date in the config to run the download. (I tried 2019/1/1 -> 2019/3/1 which is downloadable)

atangfan avatar Apr 01 '19 09:04 atangfan

but looks like I can only shorten the start_date and end_date in the config to run the download. (I tried 2019/1/1 -> 2019/3/1 which is downloadable)

Same here. I also tried shortening the period, and it worked. What would you think would be the best way to download for a longer period? To download manually multiple times, each time for a short period?

johannes73 avatar Apr 01 '19 14:04 johannes73

Same here. How to set resolution to auto? No resolution field in net_config.json. In https://github.com/ZhengyaoJiang/PGPortfolio/blob/master/user_guide.md, author mentioned that

Note that using the this file, you shouldn't make any changes to input data configuration(For example start_date, end_date or coin_number) otherwise incorrect result might be presented.

but looks like I can only shorten the start_date and end_date in the config to run the download. (I tried 2019/1/1 -> 2019/3/1 which is downloadable)

If you follow the code, you will find that there is a update_data method in the globaldatamatrix.py which will alter the end and period. The reason I think is that the local database have the former data and it will influence the parse of the current data you want to training. This is a bug of this code I think.

Solution: Just delete the Data.db, then training the data online!!! or make sure the data of the asset that u want training is not appeared in the Data.db.

This is my thinking, and it run well in my machine, hope you guys will have a better way to solve this.

JayWu7 avatar Apr 03 '19 02:04 JayWu7

but looks like I can only shorten the start_date and end_date in the config to run the download. (I tried 2019/1/1 -> 2019/3/1 which is downloadable)

Same here. I also tried shortening the period, and it worked. What would you think would be the best way to download for a longer period? To download manually multiple times, each time for a short period?

After doing the things I said above, I can train the period as long as I want.

JayWu7 avatar Apr 03 '19 02:04 JayWu7

Hi everyone, can anybody tell me where I can get the data to run this repository please? I'm using "python main.py --mode=download_data" and it gets stuck after 121K.

Thanks.

pbezz1 avatar Aug 27 '19 19:08 pbezz1

Can anyone offer a complete Data.db? I think the main cause of this error is due to my internet connection. Even I followed the instructions mentioned above.

myknowntime avatar Sep 26 '19 07:09 myknowntime

there is one data.db at release

dexhunter avatar Nov 22 '19 13:11 dexhunter

I have experienced this TypeError "TypeError: string indices must be integers" with recent dev branch repo. My json config market is set 'binance'. So, I spotted there is market default hardcoded to poloniex in datamatrices.py in DataMatrices class in init constructor.

This market value is not defined so it is not overwritten in main.py in line 71. there shall be market value added to read from config.

0xdarkman avatar Feb 14 '20 23:02 0xdarkman

Hello everyone, can anyone download the data from 2018 to 2020? I may not be able to download it due to network problems. If someone downloads it, can you share it, thank you.

yyqxnice avatar Aug 25 '21 10:08 yyqxnice