yahooquery icon indicating copy to clipboard operation
yahooquery copied to clipboard

Inconsistent data type of history

Open impredicative opened this issue 4 years ago • 4 comments
trafficstars

Describe the bug I expect history to be a dataframe. Sometimes I get a dict, and sometimes a dataframe, making it grossly inconsistent and unusable.

To Reproduce

yq.__version__
Out[1]: '2.2.15'

type(yq.Ticker('ANEW').history(period='1m', interval='1m', adj_ohlc=True))
Out[2]: dict

type(yq.Ticker('ANEW').history(period='2m', interval='1m', adj_ohlc=True))
Out[3]: dict

type(yq.Ticker('ANEW').history(period='10m', interval='1m', adj_ohlc=True))
Out[4]: pandas.core.frame.DataFrame

yq.Ticker('ANEW').history(period='10m', interval='1m', adj_ohlc=True)
Out[5]: 
                                 open       high  volume      close        low
symbol date                                                                   
ANEW   2021-03-26 15:55:00  42.169998  42.198002     0.0  42.198002  42.169998

Expected behavior I expect consistent return data types.

Additional context The package looks to be unusable for meaningfully retrieving historical data. It doesn't even seem to be enforcing basic type consistency checking via mypy.

impredicative avatar Mar 27 '21 17:03 impredicative

Hi, I am also experiencing this. Basically I found that when the ticker has no data (take COIN for example) it does NOT become an invalid ticker, instead a dictionary is returned. What you can do is to loop through the dictionary and filter out tickers that correspond to dataframe inside. I'm sure there are cleaner solutions out there but this should fix the problem you mentioned

`

df = tickers_obj.history(start=start_date, end=end_date)

if isinstance(df, dict):
    print("download_yahooquery_data:: dict returned instead of df. parsing dict to assemble_df")

    assembled_df = pd.DataFrame()
    for key, val in df.items():

        if "Data doesn't exist" not in val:
            val['symbol'] = key

                # we only want the val that is df
            if isinstance(val, pd.DataFrame):

                val = val.reset_index().rename(columns={"index": 'date'})
                val = val.set_index(['symbol', 'date'])
                assembled_df = pd.concat([assembled_df, val])
            else:
                # here the sub datafame is a dict, meaning theres no price data so add key to failed_tickers
                failed_tickers.append(key)
        else:
            failed_tickers.append(key)

    df = assembled_df.copy()`

darkknight9394 avatar May 20 '21 04:05 darkknight9394

@impredicative that's right, for single ticker one can put another if to check the dict isn't empty

darkknight9394 avatar May 21 '21 15:05 darkknight9394

@dpguthrie Think this can be fixed so users don't have to hack their way around it? Right now it is making the functionality unusable. Thanks.

impredicative avatar May 21 '21 15:05 impredicative

Check if this solves it for you: issue https://github.com/dpguthrie/yahooquery/issues/97#issuecomment-977467565 and pull request https://github.com/dpguthrie/yahooquery/pull/102

danilogalisteu avatar May 26 '22 13:05 danilogalisteu

Closed with #118

dpguthrie avatar Oct 15 '22 19:10 dpguthrie