yfinance
yfinance copied to clipboard
ranaroussi
`Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True` when `ignore_tz=False`
Describe bug
I get the error Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True when downloading history data in PROD.
My PROD setup:
- Linux AMD 64
- Timezone: CET
- Python 3.11.5
- yfinance 0.2.35
- pandas 2.1.4
Simple code that reproduces your problem
~~I cannot reproduce the error in my DEV setup, but~~ the pseudo code leading to this error, is the following:
yf.download(
tickers=" ".join(symbols),
period=period.value,
interval=interval.value,
group_by="tickers",
prepost=True,
ignore_tz=False
)
My DEV setup:
- Mac ARM 64
- Timezone: CET
- Python: 3.11.6
- yfinance 0.2.35
- pandas 2.1.4
When setting ignore_tz=True the error disappears (leading to another error that's not covered by this issue here).
Debug log
File "/home/backend/yahoo/utils/__init__.py", line 41, in __init__
self.__data = yf.download(
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yfinance/utils.py", line 103, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/yfinance/multi.py", line 225, in download
data.index = _pd.to_datetime(data.index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 1121, in to_datetime
result = convert_listlike(arg, format, name=arg.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pandas/core/tools/datetimes.py", line 490, in _convert_listlike_datetimes
result, tz_parsed = objects_to_datetime64ns(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pandas/core/arrays/datetimes.py", line 2346, in objects_to_datetime64ns
result, tz_parsed = tslib.array_to_datetime(
^^^^^^^^^^^^^^^^^^^^^^^^
File "tslib.pyx", line 403, in pandas._libs.tslib.array_to_datetime
File "tslib.pyx", line 552, in pandas._libs.tslib.array_to_datetime
File "tslib.pyx", line 480, in pandas._libs.tslib.array_to_datetime
File "conversion.pyx", line 716, in pandas._libs.tslibs.conversion.convert_timezone
ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True, at position 4
Bad data proof
No response
yfinance version
0.2.35 Previous tested version 0.2.32 works fine.
Python version
3.11.5
Operating system
Linux 6c9a93488545 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 GNU/Linux Podman 3.4.4
I was now able to reproduce this in my DEV setup. It seems to be related to specific symbols. I'm investigating. The change came with 4a5616d5c40da3dcd1350282329869fe19aea6e9
Just a thought: does this error occur when tickers from different timezones? Mixing timezones can cause problems in Pandas. Easily solved though.
Another thought: does downgrading Pandas to below 2.1 help? That introduced a subtle datetime bug that can propagate and cause chaos.
I'm experiencing this bug again in Pandas.
In multi.py line 214
data.index = _pd.to_datetime(data.index)
where data.index is:
Index([2024-08-07 09:05:00+02:00, 2024-08-07 10:40:00+02:00,
2024-08-07 13:15:00+02:00, 2024-08-07 13:20:00+02:00,
2024-08-07 09:30:00-04:00, 2024-08-07 09:35:00-04:00,
2024-08-07 09:40:00-04:00, 2024-08-07 09:45:00-04:00,
2024-08-07 09:50:00-04:00, 2024-08-07 09:55:00-04:00,
2024-08-07 10:00:00-04:00, 2024-08-07 10:05:00-04:00,
2024-08-07 10:10:00-04:00, 2024-08-07 10:15:00-04:00,
2024-08-07 10:20:00-04:00, 2024-08-07 10:25:00-04:00,
2024-08-07 10:30:00-04:00, 2024-08-07 10:35:00-04:00,
2024-08-07 10:40:00-04:00, 2024-08-07 10:45:00-04:00,
2024-08-07 10:50:00-04:00, 2024-08-07 10:55:00-04:00,
2024-08-07 11:00:00-04:00, 2024-08-07 11:05:00-04:00,
2024-08-07 11:10:00-04:00, 2024-08-07 11:15:00-04:00,
2024-08-07 11:20:00-04:00, 2024-08-07 11:25:00-04:00,
2024-08-07 11:30:00-04:00, 2024-08-07 11:35:00-04:00,
2024-08-07 11:40:00-04:00, 2024-08-07 11:45:00-04:00,
2024-08-07 11:50:00-04:00, 2024-08-07 11:55:00-04:00,
2024-08-07 12:00:00-04:00, 2024-08-07 12:05:00-04:00,
2024-08-07 12:10:00-04:00, 2024-08-07 12:15:00-04:00,
2024-08-07 12:20:00-04:00, 2024-08-07 12:25:00-04:00,
2024-08-07 12:30:00-04:00, 2024-08-07 12:35:00-04:00,
2024-08-07 12:40:00-04:00, 2024-08-07 12:45:00-04:00,
2024-08-07 12:50:00-04:00],
dtype='object')
It fails with:
Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True, at position 4
Position 4 has a timezone of -4.