Download with 60d intraday leads to internal error

[kschmid]

Describe bug

60d timeframe sometimes leads to an exception for downloading. For some symbols like COP (not sure, what differentiates the concerned symbols), I get an internal error: 1 Failed download: ['COP']: IndexError('Boolean index has wrong length: 2 instead of 1') The corresponding code: parthist = yf.download(ticker_name, period="60d", interval="2m", session=mysession)

If I change the 60d to 30d, everything works apparently.

Simple code that reproduces your problem

parthist = yf.download(ticker_name, period="60d", interval="2m", session=mysession)

Debug log

DEBUG Entering download() DEBUG Disabling multithreading because DEBUG logging enabled DEBUG Entering history() DEBUG COP: Yahoo GET parameters: {'range': '60d', 'interval': '2m', 'includePrePost': False, 'events': 'div,splits,capitalGains'} DEBUG COP: yfinance received OHLC data: 2023-08-25 13:30:00 -> 2023-10-06 20:00:00 DEBUG COP: OHLC after cleaning: 2023-08-25 09:30:00-04:00 -> 2023-10-06 15:58:00-04:00 ERROR 1 Failed download: ERROR ['COP']: IndexError('Boolean index has wrong length: 2 instead of 1') DEBUG ['COP']: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/yfinance/multi.py", line 275, in _download_one data = Ticker(ticker, session=session).history( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/yfinance/utils.py", line 114, in wrapper result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/yfinance/base.py", line 373, in history df = utils.safe_merge_dfs(df, dividends, interval) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/yfinance/utils.py", line 740, in safe_merge_dfs raise Exception(f"The following '{data_col}' events are out-of-range, did not expect with interval {interval}: {df_sub.index[f_outOfRange]}") ~~~~~~~~~~~~^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 5382, in getitem result = getitem(key) ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/arrays/datetimelike.py", line 376, in getitem result = cast("Union[Self, DTScalarOrNaT]", super().getitem(key)) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/arrays/_mixins.py", line 284, in getitem key = check_array_indexer(self, key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pandas/core/indexers/utils.py", line 539, in check_array_indexer raise IndexError( IndexError: Boolean index has wrong length: 2 instead of 1

DEBUG Exiting download() Empty DataFrame Columns: [Open, High, Low, Close, Adj Close, Volume] Index: [] success2 DEBUG Entering download() DEBUG Disabling multithreading because DEBUG logging enabled DEBUG Entering history() DEBUG COP: Yahoo GET parameters: {'range': '7d', 'interval': '1m', 'includePrePost': False, 'events': 'div,splits,capitalGains'} DEBUG COP: yfinance received OHLC data: 2023-09-28 13:30:00 -> 2023-10-06 20:00:00 DEBUG COP: OHLC after cleaning: 2023-09-28 09:30:00-04:00 -> 2023-10-06 15:59:00-04:00 DEBUG COP: OHLC after combining events: 2023-09-28 09:30:00-04:00 -> 2023-10-06 15:59:00-04:00 DEBUG COP: yfinance returning OHLC: 2023-09-28 09:30:00-04:00 -> 2023-10-06 15:59:00-04:00 DEBUG Exiting history() DEBUG Exiting download()

Bad data proof

No response

yfinance version

0.2.31

Python version

Python 3.11.5 (main, Aug 24 2023, 15:18:16) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin

Operating system

MacOS Ventura 13.6

[ValueRaider]

Merging dividends/stock-splits with intraday price data is a pain. So maybe two things needs to be implemented:

• fix this bug
• add tolerance for merge failures with intraday prices because probably you don't care about dividends/stock-splits

[kschmid]

I see, so this is the root. Yes, this merging is a a pain.

[rickturner2001]

@ValueRaider is there a bug to fix though? I guess I don't understand why we are trying to merge two incompatible dataframes. Maybe someone who's interested can shed some lights on this

[ValueRaider]

"Incompatible" is unfair, they're both time-series data. Dividends have been merged with intraday prices since before I began contributing, or at least a merge attempted. Just needs a little Numpy adjustment.