yfinance icon indicating copy to clipboard operation
yfinance copied to clipboard

Agregate (and accelerate) getting stock info from a list of tickers

Open lcrmorin opened this issue 2 years ago • 1 comments

I have been able to aggregate stock info from different tickers by using a custom aggregation function (see below). But this operation is very slow. Running this on a decent list of tickers takes hours (here 2000 tickers for the JPX competition) . To be fair the problem seems to come from the data loading time.

Any way to make this faster ?


def transform_list_to_df(list, test=False):
    formated_list = []
    for i in dict_t.items():
        if i[1] is None:
            formated_list.append((i[0],i[1]))
        elif type(i[1]) in [str,bool,float,int]:
            formated_list.append((i[0],i[1]))
        elif type(i[1]) in [dict]:    
            for j in i[1]:
                formated_list.append((i[0]+'_'+j,i[1][j]))
        else:
            if i[0]=='companyOfficers':
                if test:
                    print(i[1])
            elif i[0]=='sectorWeightings':
                for j in i[1]:
                    for k in j:
                        formated_list.append(('w_'+k,j[k]))
            elif i[0]=='holdings':
                if test:
                    print(i[1])
            elif i[0]=='bondRatings':
                for j in i[1]:
                    for k in j:
                        formated_list.append((i[0]+'_'+k,j[k]))
                         
    df_t = pd.DataFrame(formated_list)
    df_t = df_t.set_index(0)
    df_t.columns = [ticker_name]
    
    return(df_t.T)

df = pd.DataFrame()

for ticker_name in tickers:
    if DEBUG:
        print(ticker_name)
    t =  yf.Ticker(ticker_name)
    dict_t = t.info
    df = pd.concat((df,transform_list_to_df(dict_t).copy()))

CPU times: user 14min 4s, sys: 22.3 s, total: 14min 26s Wall time: 6h 20min 40s

lcrmorin avatar May 15 '22 11:05 lcrmorin

The solution is to cache locally. Have you tried using a requests session object? It's described in Quick Start.

ValueRaider avatar Jun 04 '22 13:06 ValueRaider