FinRL icon indicating copy to clipboard operation
FinRL copied to clipboard

Finrl Meta Stock_NeurIPS2018_SB3.ipynb only support DOW_30_TICKER

Open darenwai opened this issue 1 year ago • 6 comments

Describe the bug using : df = YahooDownloader(start_date = TRAIN_START_DATE, end_date = TRADE_END_DATE, ticker_list = config_tickers.DOW_30_TICKER).fetch_data() df.to_csv('sample.csv') df = pd.read_csv('sample.csv') i am able to generate FeatureEngineer technical indicator

So i generate a csv file with my own stock of malaysia market in the exact format from the above sample.csv. i got error when run this: fe = FeatureEngineer( use_technical_indicator=True, tech_indicator_list = INDICATORS, use_vix=False, use_turbulence=False, user_defined_feature = False) KeyError: "None of [Index(['tic', 'date', 'boll_ub'], dtype='object')] are in the [columns]"

I attached my sample.csv for anyone who can take a look. The error persist even after disabling vix as it use yahoo downloader to download vis. How to adapt this notebook to work with my own stocks instead?

sample1try.csv

darenwai avatar Jul 31 '23 07:07 darenwai

To address your issue:

The tickers you're interested in are from the Malaysia stock market. From Yahoo Finance, Malaysia tickers include CMSB.KL, ALCOM.KL, PPHB.KL, and PRLEXUS.KL. You can find more tickers from the Malaysia market that you can use in your code on Yahoo Finance.

You would replace the ticker_list with, and you can add more tickers to the list:

MALAYSIA_TICKERS = ['CMSB.KL', 'ALCOM.KL', 'PPHB.KL', 'PRLEXUS.KL']

And change the line:

ticker_list = config_tickers.DOW_30_TICKER

to:

df = YahooDownloader(start_date = TRAIN_START_DATE,
end_date = TRADE_END_DATE,
ticker_list = MALAYSIA_TICKERS).fetch_data()

mmmarchetti avatar Aug 24 '23 13:08 mmmarchetti

Have you actually tried MALAYSIA_TICKERS = ['CMSB.KL', 'ALCOM.KL', 'PPHB.KL', 'PRLEXUS.KL']. I have tried and it doesnt work. Anyway my problem is not the data. I have my own ticker data. When i run the sample, there is error. I fixed the problem by ensuring all ticker must have same number of rows which is silly because some ticker have missing trading days due to various reason. Eventhough i have successfully running the sample also no use. The result is the drl agent buy everyday non stop for 7months wihout selling in the out of sample results. Obviously the drl agent got something wrong eventhough i have sucessfully train the agent.

darenwai avatar Aug 24 '23 14:08 darenwai

Upon investigating the issue, I found that the FeatureEngineer function expects the date format to be of type str (or object in pandas terms) rather than a pandas datetime64 type. This is likely because the function internally processes or compares the date values as strings.

To remedy this, you can convert the date column back to a string format before passing the DataFrame to FeatureEngineer. Here's how you can do it:

df['date'] = df['date'].dt.strftime('%Y-%m-%d')

This code will convert the datetime64 column back to a string format in the 'YYYY-MM-DD' style.

After this conversion, you can then use the FeatureEngineer function as intended:

FeatureEngineer(use_technical_indicator=True,
                tech_indicator_list=config.INDICATORS,
                use_vix=True,
                use_turbulence=True,
                user_defined_feature=False)

I understand that this might not be the most intuitive requirement, but it solves the problem.

mmmarchetti avatar Aug 24 '23 16:08 mmmarchetti

Thank you so much. I will try it out and let you know the result. Thanks

darenwai avatar Aug 25 '23 00:08 darenwai

Hi. I already tried your method using the above sample1try.csv and run it at FinRL-master/examples/Stock_NeurIPS2018_1_Data.ipynb and i got this error: KeyError: "None of [Index(['tic', 'date', 'macd'], dtype='object')] are in the [columns]"

You able to run the above excel without problem?

darenwai avatar Aug 25 '23 02:08 darenwai

Hello. I encountered the following problem: df = YahooDownloader(start_date=TRAIN_START_DATE, end_date=TEST_END_DATE, ticker_list=DOW_30_TICKER).fetch_data() print(df) An error appears: [100%%*] 1 of 1 completed Traceback (most recent call last): File "C:\Users\LIKEP\PycharmProjects\trading analyzes\main.py", line 52, in df = YahooDownloader(start_date=TRAIN_START_DATE, end_date=TEST_END_DATE, ticker_list=DOW_30_TICKER).fetch_data() File "C:\Users\LIKEP\PycharmProjects\trading analys\venv\lib\site-packages\finrl\finrl_meta\preprocessor\yahoodownloader.py", line 51, in fetch_data data_df = data_df.append(temp_df) File "C:\Users\LIKEP\PycharmProjects\trading analys\venv\lib\site-packages\pandas\core\generic.py", line 6293, in getattr return object.getattribute(self, name) AttributeError: 'DataFrame' object has no attribute 'append'

Lenar76 avatar Jan 29 '24 01:01 Lenar76