Failed downloads - "No timezone found, symbol may be delisted"

[waterflyer]

Hi yfinance team, start_date = '2020-01-02' end_date = '2022-01-01' ticker_list = ['META', 'GOOG', 'ORCL', 'MSFT', 'AMZN'] data = yf.download(ticker_list, start=start_date, end=end_date, ignore_tz=True)[['Close']]

I'm using the above code snippet to download 5 tickers' data, but I got the following error messages no matter if I changed ignore_tz to False or True.

[100%**] 5 of 5 completed

5 Failed downloads:

• AMZN: No timezone found, symbol may be delisted
• META: No timezone found, symbol may be delisted
• GOOG: No timezone found, symbol may be delisted
• MSFT: No timezone found, symbol may be delisted
• ORCL: No timezone found, symbol may be delisted

yfinance version: 0.2.9 python version: 3.8.5 Operating system: Azure Machine Learning Studio or my kernel is Python 3.8 - AzureML

Please help me figure it out. I really appreciate any help you can provide.

[ValueRaider]

ignore_tz only affects when combining the price tables. Still need timezones to localize the dates.

Run this and report output:

for tkr in ticker_list:
	dat = yf.Ticker(tkr)
	tz = dat._fetch_ticker_tz(debug_mode=True, proxy=None, timeout=30)
	valid = yf.utils.is_valid_timezone(tz)
	print(f"{tkr}: tz='{tz}', valid={valid}")

[waterflyer]

@ValueRaider Thank you for your response, as you suggested, after testing, this is the output

[vidalmarco]

@waterflyer Are you behind a proxy?

[vidalmarco]

@ValueRaider Not sure if it can help but I had the same issue even with a simple download() call: yf.download(ticker, start=start_date, end=end_date) and I've noticed it was something related to tz retrieval from cache when proxy is in the picture.

So, I run the code you sent above against my ticker list by specifying the proxy string instead of None (as I'm behind a proxy). It gave True for every ticker's timezone and since then also the download of the data started to work AMZN: tz='America/New_York', valid=True TSLA: tz='America/New_York', valid=True UBS: tz='America/New_York', valid=True CS: tz='America/New_York', valid=True GOOG: tz='America/New_York', valid=True GOOGL: tz='America/New_York', valid=True DAX: tz='America/New_York', valid=True

yFinance: 0.2.9 + hotfix/proxy (base.py and fundamentals.py) python: 3.9.12 OS: Windows 10 20H2

[vidalmarco]

Update from my side. I was trying to d/l new indices price history when I got the same issue. Here below the output with the check_validity code from @ValueRaider at the end:

0.2.9 Download the historical stock prices for ^GSPC: 1 Failed download:

• ^GSPC: No timezone found, symbol may be delisted

Download the historical stock prices for ^DJI: 1 Failed download:

• ^DJI: No timezone found, symbol may be delisted

Download the historical stock prices for ^IXIC: 1 Failed download:

• ^IXIC: No timezone found, symbol may be delisted

Download the historical stock prices for ^NYA: 1 Failed download:

• ^NYA: No timezone found, symbol may be delisted

^GSPC: tz='America/New_York', valid=True ^DJI: tz='America/New_York', valid=True ^IXIC: tz='America/New_York', valid=True ^NYA: tz='America/New_York', valid=True

I've got 2 PC: one is behind proxy and one not. The one not behind a proxy works fine with this output: yFinance ver.: 0.2.9 Download the historical stock prices for ^GSPC: 504 observation dowloaded. Download the historical stock prices for ^DJI: 504 observation dowloaded. Download the historical stock prices for ^IXIC: 504 observation dowloaded. ^GSPC: tz='America/New_York', valid=True ^DJI: tz='America/New_York', valid=True ^IXIC: tz='America/New_York', valid=True

I debugged this issue and found that all is due to the fact that tz for the new tickers (e.g. ^GSPC) is not cached. As a consequence a new query is submitted (base.py _fetch_ticker_tz()) in order to retrieve it from yahoo and to cache it. For some reason the query fails and the tz it is not written into the cache (which is a SQLite3 db in C:\Users<username>\AppData\Local\py-yfinance\tkr-tz.db under Win OS). I'm still investigating the root cause behind the query failure but considering that on my PC not behind a proxy, it works I'm suspecting this is due to a non correct usage of the proxy in the _fetch_ticker_tz() query.

Workaround As a workaround I've just edited the kv table of cache db (e.g. with SQLiteStudio and manually added the row for ^GSPC and indeed it works.

HTH

[ValueRaider]

I'm suspecting this is due to a non correct usage of the proxy

Great clue. The timezone fetch was happening before yfinance had prepared the proxy. Can you try branch fix/fetch-tz-via-proxy

[vidalmarco]

Final update from my side.

I can confirm the issue is due to the format of proxy which goes as input to the web query base.py _fetch_ticker_tz().

Such kind of queries require a string for the proxy but in input the proxy information is a dict.

Therefore a code like this one (which has been already used elesewhere) in line 1209 in base.py _fetch_ticker_tz() after the try statement ensures it is a string and fix the issue:

if proxy is not None:
    if isinstance(proxy, dict) and "https" in proxy:
       proxy = proxy["https"]

hope it will be integrated into hotfix/proxy soon.

Be aware that in your main codes proxies has to be declared as dictionary. E.g.: my_proxy={'http':'<ip-address>:<port>', 'https':'<ip-address>:<port>'} yf.dowload(ticker, start=my_start, end=my_end, proxy=my_proxy)

[vidalmarco]

@ValueRaider I read now your message :-). Let me try your fix...

[vidalmarco]

I've tried the _fix/fetch-tz-via-proxy_ and removed the last row of the block as it'd reformat proxy info as a dict. But we need a string here. Now, it works well 👍 thnx. I've tested it with both with existing and new indexes tickers and works well. Enjoy!

[ValueRaider]

removed the last row of the block as it'd reformat proxy info as a dict

But yfinance has always put in a dict. What happens if you leave it?

[vidalmarco]

I see what you mean... Indeed all the data request are demanded to TickerData object which ensures correct proxy format with get() and _get_proxy() methods. I'll make more tests. Stay tuned.

[ValueRaider]

Can you switch to branch hotfix/proxy and run the new test: tests/ticker.py :: test_goodTicker_withProxy. Obviously set a proxy in setUpClass.

[vidalmarco]

In the meantime I did the following test by using old-style debug approach and insert print statements in base.py and data.py (To be noted that to test we need a brand new ticker never cached before). For a cached ticker both codes (i.e. with or without the last line) work.

Bottom line Differences seem related to frozendict usage behind the scene.... and removing the last line seems indeed to be just a side-effect workaround

With the last line:

    # setup proxy in requests format
    if proxy is not None:
        if isinstance(proxy, dict) and "https" in proxy:
            proxy = proxy["https"]
        proxy = {"https": proxy}

Output: Download the historical stock prices for ^N225:

base.py, TickerBase:history():after fix/fetch_tx_via_proxy block:proxy: {'https': '10.xx.xxx.xx:pppp'} base.py, TickerBase:_fetch_ticker_tz():before _data.cache_get:proxy: {'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get:before proxy reformat:proxy--> frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'}) data.py,TickerData::get:after proxy reformat and before _session.get():proxy--> {'https': frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'})}

1 Failed download: ^N225: No timezone found, symbol may be delisted

Without the last line:

    # setup proxy in requests format
    if proxy is not None:
        if isinstance(proxy, dict) and "https" in proxy:
            proxy = proxy["https"]

Output Download the historical stock prices for ^N225:

base.py, TickerBase:history():after fix/fetch_tz_via_proxy block:proxy: 10.xx.xxx.xx:pppp base.py, TickerBase:_fetch_ticker_tz():before _data.cache_get-->proxy: 10.xx.xxx.xx:pppp data.py,TickerData::get():before proxy reformat:proxy--> 10.xx.xxx.xx:pppp data.py,TickerData::get():after proxy reformat and before _session.get():proxy--> {'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get():before proxy reformat:proxy--> 10.xx.xxx.xx:pppp data.py,TickerData::get():after proxy reformat and before _session.get():proxy--> {'https': '10.xx.xxx.xx:pppp'}

490 observation dowloaded.

[vidalmarco]

[ValueRaider]

Run this: Ticker(tkr)._fetch_ticker_tz(debug_mode=True, proxy=proxy, timeout=10) It should print any errors occurring in the get.

[vidalmarco]

sorry, where should I run it? dat object I guess is a TickerData instance... shall I insert this line in my test main()?

[ValueRaider]

Sorry, I've edited now

[vidalmarco]

Hope I've understood well (sorry if I left my print):

• insert: yf.Ticker(ticker)._fetch_ticker_tz(debug_mode=True, proxy=proxy, timeout=10)
• revert to "with last line" version
• run it against two indexes: FTSE and NIKKEI

HTH Output 0.2.9

base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get:before proxy reformat:proxy--> frozendict.frozendict({'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'}) data.py,TickerData::get:after proxy reformat and before _session.get():proxy--> {'https': frozendict.frozendict({'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'})}

Failed to get ticker '^N225' reason: expected string or bytes-like object

Download the historical stock prices for ^N225: base.py, TickerBase.history():after fix/fetch_tx_via_proxy block:proxy: {'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get:before proxy reformat:proxy--> frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'}) data.py,TickerData::get:after proxy reformat and before _session.get():proxy--> {'https': frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'})}

1 Failed download:

• ^N225: No data found for this date range, symbol may be delisted

base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get:before proxy reformat:proxy--> frozendict.frozendict({'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'}) data.py,TickerData::get:after proxy reformat and before _session.get():proxy--> {'https': frozendict.frozendict({'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'})}

Failed to get ticker '^FTSE' reason: expected string or bytes-like object

Download the historical stock prices for ^FTSE: base.py, TickerBase.history():after fix/fetch_tx_via_proxy block:proxy: {'https': '10.xx.xxx.xx:pppp'} base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'https': '10.xx.xxx.xx:pppp'} data.py,TickerData::get:before proxy reformat:proxy--> frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'}) data.py,TickerData::get:after proxy reformat and before _session.get():proxy--> {'https': frozendict.frozendict({'https': '10.xx.xxx.xx:pppp'})}

1 Failed download:

• ^FTSE: No timezone found, symbol may be delisted

[vidalmarco]

fyi I also run it without the last line and it works, but I get the same message "Failed to get ticker '^FTSE' reason: expected string or bytes-like object"

HTH

[vidalmarco]

IMHO from a cleaner design I think you are right and we should try to leave the last line. The root cause seems to be in another point and related to frozendict. Removing the last line and pass proxy as a string appears to be a workaround...

[ValueRaider]

I've pushed a potential fix, try again.

[vidalmarco]

replaced data.py with your last version. Still not working...

Output 0.2.9 base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'} Failed to get ticker '^N225' reason: type object 'frozendict.frozendict' has no attribute 'frozendict' Download the historical stock prices for ^N225: base.py, TickerBase.history():after fix/fetch_tx_via_proxy block:proxy: {'https': '10.xx.xxx.xx:pppp'} base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'https': '10.xx.xxx.xx:pppp'}

1 Failed download: ^N225: No timezone found, symbol may be delisted

[ValueRaider]

type object 'frozendict.frozendict' has no attribute 'frozendict'

Fixed.

[vidalmarco]

Output 0.2.9 base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'http': '10.xx.xxx.xx:pppp', 'https': '10.xx.xxx.xx:pppp'} Failed to get ticker '^N225' reason: expected string or bytes-like object Download the historical stock prices for ^N225: base.py, TickerBase.history():after fix/fetch_tx_via_proxy block:proxy: {'https': '10.xx.xxx.xx:pppp'} base.py, TickerBase:_fetch_ticker_tz:before _data.cache_get-->proxy: {'https': '10.xx.xxx.xx:pppp'}

1 Failed download: ^N225: No timezone found, symbol may be delisted

[vidalmarco]

Just to remark that I'm testing with the last base.py in the hotfix/proxy branch modified with uncommented line 514: proxy = {"https": proxy}

[ValueRaider]

I can't keep debugging like this, via you. Can you take over? Focus on the get function, maybe the other arguments need to be converted frozendict -> dict

[vidalmarco]

ok.

[vidalmarco]

may I have write access to the code hotfix/proxy branch as a contributor?

[ValueRaider]

I can't modify repo permissions. You can fork and pull request.

[vidalmarco]

ok. Noted for the future 👍 Thanks. Please, modify _get_proxy() method as follows and it will work:

    def _get_proxy(self, proxy):
        # setup proxy in requests format
        if proxy is not None:
            if (isinstance(proxy, dict) or isinstance(proxy, frozendict)) and "https" in proxy:
                proxy = proxy["https"]
            proxy = {"https": proxy}
        return proxy