`inputs are all NaN` causes `input_arrays parameter missing required data key: ema1`

[tesla-cat]

problem

• if some talib function is called and produces some error, and this is handled by try-except, then the subsequent calls to this or other talib functions gives errors. However, by removing the first call that causes the first error, there are no more subsequent errors

simplified scenario

the following is a made-up scenario to demonstrate the idea, because the original code producing the error has heavy dependencies that can not be shown here

import talib.abstract as ta

df = pd.DataFrame()
for T in [10, 20, 30]:
    try:
        sma1 = ta.SMA(df, T)
    except Exception as e:
        print(f'ERROR: {len(df)} {e}')

• if for T=10 there is an error inputs are all NaN
• then for T=20, T=30 there will be input_arrays parameter missing required data key: ema1
• however, if remove 10 and only keep [20, 30], there are no input_arrays parameter missing required data key: ema1
• this means somehow, the error produced when calling some talib function for the first time, influences the outcome of consequent function calls, even though the first error is handle by try-except, by removing the first call, the following calls produce no errors

original code

• refer to freqtrade

def popIndicators(st: IStrategy, dp: DataProvider, interval='5m'):
    dfs: dict[str, pd.DataFrame] = {}
    msgs = []
    for pair, df in dp.data[interval].items():
        try:
            dfs[pair] = st.populate_indicators(df.copy(), {'pair':pair})
        except Exception as e:
            msgs.append(f'ERROR popIndicators: {st.name} {interval} {pair} {len(df)} {e}')
    return dfs, msgs

• partial results:

    "Combined_NFIv7_SMA": [
        "ERROR popIndicators: Combined_NFIv7_SMA 5m FIDAUSDT 49 inputs are all NaN",
        "ERROR popIndicators: Combined_NFIv7_SMA 5m FILUSDT 8617 input_arrays parameter missing required data key: ema1",
        "ERROR popIndicators: Combined_NFIv7_SMA 5m FIOUSDT 8617 input_arrays parameter missing required data key: ema1",
        "ERROR popIndicators: Combined_NFIv7_SMA 5m FIROUSDT 8617 input_arrays parameter missing required data key: ema1",
        "ERROR popIndicators: Combined_NFIv7_SMA 5m FISUSDT 8617 input_arrays parameter missing required data key: ema1",
        ...

[mrjbq7]

I can't quite reproduce that yet with your example, I got this with an empty data frame:

ERROR: 0 input_arrays parameter missing required data key: close
ERROR: 0 input_arrays parameter missing required data key: close
ERROR: 0 input_arrays parameter missing required data key: close

And this with one that contains all NaNs:

ERROR: 1 inputs are all NaN
ERROR: 1 inputs are all NaN
ERROR: 1 inputs are all NaN

But it's possible that throwing an exception leaves the Abstract API wrapper in a mixed-up state. I will investigate further.

[tesla-cat]

thanks, I wrote the simplified scenario just to demonstrate the idea, because the original code depends heavily on freqtrade and thus may not make sense to you

[mrjbq7]

Hmm, I added a test case in d4e8b0c4ab3900b0ba798fa356e7ad4e27dbac77 and it doesn't have your issue.

Some thoughts about your scenario...

The abstract interface saves the previous inputs for subsequent calls, mostly for convenience, see for example:

>>> from talib import abstract

>>> inputs = {'close': np.array([1.0, 2.0, 3.0])}

>>> abstract.SMA(inputs, timeperiod=2)
array([nan, 1.5, 2.5])

# this uses the previous provided inputs, but with a different timeperiod
>>> abstract.SMA(timeperiod=3)
array([nan, nan,  2.])

Are you accessing TA-Lib from multiple threads?

Do you think the first exception ("inputs are all NaN") is correct, but the subsequent ones are not?