backtesting.py icon indicating copy to clipboard operation
backtesting.py copied to clipboard

TypeError: Index.get_loc() got an unexpected keyword argument 'method'

Open luludocteur opened this issue 11 months ago • 8 comments

Hi everybody,

I'm kind of a new user of backtesting.py. I've coded a litlle strategy of grid trading of BTC in 1 minute. I make use of the ATR indicator used for Stop Loss that i resampled in daily. My problem is linked to the plot() method. Indeed my code is working well because i can print the Statistics with stat = bt.run().

If you have any questions, feel free to ask, i'll be pleased to answer you. Thank you

Expected Behavior

Running the strategy and printing the plot

Actual Behavior

When the code launch the plot() method, it launch the _plotting.py program. I let you check the whole error but the last problem is the exact title of this topic : "TypeError: Index.get_loc() got an unexpected keyword argument 'method'". I digged a little into the code and it seems that this argument is expected in the _group_trades function:

  def _group_trades(column):
    def f(s, new_index=pd.Index(df.index.view(int)), bars=trades[column]):
        if s.size:
            # Via int64 because on pandas recently broken datetime
            mean_time = int(bars.loc[s.index].view(int).mean())
            new_bar_idx = new_index.get_loc(mean_time, method='nearest')
            return new_bar_idx
    return f

The whole traceback:

/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:122: UserWarning: Data contains too many candlesticks to plot; downsampling to '5T'. See `Backtest.plot(resample=...)`
  warnings.warn(f"Data contains too many candlesticks to plot; downsampling to {freq!r}. "
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:126: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  df = df.resample(freq, label='right').agg(OHLCV_AGG).dropna()
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:128: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  indicators = [_Indicator(i.df.resample(freq, label='right').mean()
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:136: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  equity_data = equity_data.resample(freq, label='right').agg(_EQUITY_AGG).dropna(how='all')
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:153: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
  trades = trades.assign(count=1).resample(freq, on='ExitTime', label='right').agg(dict(
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py:147: FutureWarning: Series.view is deprecated and will be removed in a future version. Use ``astype`` as an alternative to change the dtype.
  mean_time = int(bars.loc[s.index].view(int).mean())
Traceback (most recent call last):
  File "/Users/alexanderlunel/Documents/Python/Bots/grid_bot/btmodele.py", line 99, in <module>
    bt.plot()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/backtesting.py", line 1592, in plot
    return plot(
           ^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 203, in plot
    df, indicators, equity_data, trades = _maybe_resample_data(
                                          ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 153, in _maybe_resample_data
    trades = trades.assign(count=1).resample(freq, on='ExitTime', label='right').agg(dict(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/resample.py", line 352, in aggregate
    result = ResamplerWindowApply(self, func, args=args, kwargs=kwargs).agg()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 190, in agg
    return self.agg_dict_like()
           ^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 423, in agg_dict_like
    return self.agg_or_apply_dict_like(op_name="agg")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like
    result_index, result_data = self.compute_dict_like(
                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/apply.py", line 497, in compute_dict_like
    getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate
    return self._python_agg_general(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general
    result = self._grouper.agg_series(obj, f)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 864, in agg_series
    result = self._aggregate_series_pure_python(obj, func)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/ops.py", line 885, in _aggregate_series_pure_python
    res = func(group)
          ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 324, in <lambda>
    f = lambda x: func(x, *args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/backtesting/_plotting.py", line 148, in f
    new_bar_idx = new_index.get_loc(mean_time, method='nearest')
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Index.get_loc() got an unexpected keyword argument 'method'

Steps to Reproduce


import numpy as np 
import pandas as pd
import pandas_ta as ta
from backtesting import Strategy, Backtest
from backtesting.lib import resample_apply

df = pd.read_csv('/Users/alexanderlunel/Documents/Python/Bots/grid_bot/data.csv', sep=';')
df['Open_time'] = pd.to_datetime(df['Open_time'], unit='ms')
df.set_index(df['Open_time'], inplace = True)


#Grille
def grid(midprice, space, n):
    return np.arange(midprice-(n/2*space), midprice+(n/2*space), space)

midprice, space, n = 20000, 200, 10
grid = grid(midprice, space, n)

signal = [0]*len(df)

i = 0
for date, candle in df.iterrows():
    for p in grid:
        if p>candle.Low and p<candle.High:
            signal[i]=p
    i+=1

df['signal']=signal

def SIGNAL():
    return df.signal

class Strat(Strategy):

    nb_short = 0
    nb_long = 0
    size = 2
    maxdrawdown = 0.1
    grid = grid
    space = 200

    def init(self):
        self.signal = self.I(SIGNAL)
        self.atr = resample_apply('1D', ta.atr, self.data.High.s, self.data.Low.s, self.data.Close.s, length = 14)
        #self.atr = self.I(ta.atr, self.data.High.s, self.data.Low.s, self.data.Close.s, length = 14)
        #self.ratio = 0.05
        # daily_close = self.data.df.resample('D', label='right', on='Open_time')['Close'].agg('last')
        # daily_low = self.data.df.resample('D', label='right', on='Open_time')['Low'].agg('last')
        # daily_high = self.data.df.resample('D', label='right', on='Open_time')['High'].agg('last')
        # self.atr = self.I(ta.atr, daily_high, daily_low, daily_close, 14)

    def next(self):
        before = self.data.Close[-2]
        price = self.data.Close[-1]

        if (self.signal[-1] in self.grid):
            if len(self.trades) != 0 and self.signal[-1] != self.trades[-1].entry_price:
                if (self.nb_long_short()[0] <= 3) and (before > price):
                        slu = price - self.atr[-1]
                        tpu = price + self.space
                        self.buy(tp=tpu, sl=slu, size=self.size)
                        self.nb_long+=1
                elif (self.nb_long_short()[1] <= 3) and (before > price):
                        sld = price + self.atr[-1]
                        tpd = price - self.space
                        self.sell(tp=tpd, sl=slu,size=self.size)
                        self.nb_short+=1
            else:
                if before > price:
                    slu = price - self.atr[-1]
                    tpu = price + self.space
                    self.buy(tp=tpu, sl=sld, size=self.size)
                    self.nb_long+=1
                elif before < price:
                    sld = price + self.atr[-1]
                    tpd = price - self.space
                    self.sell(tp=tpd, sl=sld, size=self.size)
                    self.nb_short+=1
    
    def nb_long_short(self):
        long, short = 0, 0
        for trade in self.trades:
            if trade.is_long:
                long+=1
            elif trade.is_short:
                short+=1
        self.nb_long = long
        self.nb_short = short
        return self.nb_long, self.nb_short

bt = Backtest(df, Strat, cash=10_000_000, hedging = True, margin=1/10, commission=0.000, exclusive_orders=False)

stat = bt.run()
print(stat)
bt.plot()

# stats, heatmap = bt.optimize(
#     TPSLratio = list(np.arange(0.1,2,100)),
#     maximize = 'Return [%]',
#     max_tries = 200,
#     return_heatmap=True
# )[25000. 25350. 25700. 26050. 26400. 26750. 27100. 27450. 27800. 28150.]

Additional info

  • Backtesting version: 0.3.3
  • bokeh.__version__:
  • OS: MAC OS Monterey Version 12.2.1

luludocteur avatar Mar 22 '24 10:03 luludocteur

Same issue. seems like the API has changed in pandas 2.0.3

devansh-alphagrep avatar Mar 24 '24 18:03 devansh-alphagrep

Yes i agree, probably a version issue. But the fact is that it work perfectly fine when i tried to plot simple strategies available on this github. So i think it also have something to do with resampling the indicator. idk..

If you success to fix it, i'm curious

luludocteur avatar Mar 25 '24 11:03 luludocteur

@luludocteur the pypy repo hasn't been updated in a while, so it still uses the old pandas.

Just clone this repo from github and run pip install -e . in the repo to install the latest backtesting.py

devansh-alphagrep avatar Mar 25 '24 11:03 devansh-alphagrep

This is still an ongoing problem. Why don't you just update the repo ?

Best regards

Joyen12 avatar May 07 '24 10:05 Joyen12

I just put this in requirements.txt

backtesting @ git+https://github.com/kernc/backtesting.py@master

vasinl124 avatar May 13 '24 02:05 vasinl124

I just put this in requirements.txt

backtesting @ git+https://github.com/kernc/backtesting.py@master

Yeah I fixed it too directly in site-packages lol but that would still be better for future user to have something working right away without them to have to search why it's not working.

Joyen12 avatar May 14 '24 12:05 Joyen12

the last update was in 2021 soooooooo little chance it will happen, I don't know :(

vasinl124 avatar May 14 '24 15:05 vasinl124

Here is a fork in which the error does not occur. It was solved by locking old versions in the dependencies. The aim was to create a stable version. We will subsequently migrate to newer versions.

https://github.com/LUCIT-Systems-and-Development/lucit-backtesting

oliver-zehentleitner avatar Jun 06 '24 22:06 oliver-zehentleitner