pandas-datareader icon indicating copy to clipboard operation
pandas-datareader copied to clipboard

can not fetch data before 1970 on yahoo finance

Open houpuli opened this issue 3 years ago • 1 comments

Hi!
I run into an error when using pandas-datareader. I think it's probably related to the unix time stamp.

my code:

from pandas_datareader import data as web import datetime import pandas as pd import matplotlib.pyplot as plt #spy 500 index stock = '%5EGSPC' endDate = datetime.datetime(2022, 8, 30) start_date = datetime.datetime(1928, 12, 29) df = web.DataReader(stock, data_source='yahoo', start=start_date,end=endDate)'

and I get error:

'envs\stock_predict\lib\site-packages\pandas_datareader\yahoo\daily.py in _get_params(self, symbol) 124 # This needed because yahoo returns data shifted by 4 hours ago. 125 four_hours_in_seconds = 14400 --> 126 unix_start = int(time.mktime(self.start.timetuple())) 127 unix_start += four_hours_in_seconds 128 day_end = self.end.replace(hour=23, minute=59, second=59)

OverflowError: mktime argument out of range'

I tried and it works for a time after 1970, so seems it's due to converting of the UNIX time stamp.

Could you check it? Thanks!

houpuli avatar Sep 01 '22 19:09 houpuli

I came across the same issue on Fred's data and researched.

Check '4.16 Seconds Since the Epoch'. There is no standard if unit time became negative. https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html

I feel it's too hard to reverse engineer since lot's users or vendors defined conversion formula.

yosukesan avatar Jan 09 '23 06:01 yosukesan