backtesting.py icon indicating copy to clipboard operation
backtesting.py copied to clipboard

Optimize function not taking the objects passed at first

Open jsulopzs opened this issue 2 years ago • 5 comments

Expected Behavior

I expect the optimize() function to take the objects I pass to the parameters instead of the default values.

Actual Behavior

Using the following Strategy with model=None

class SimpleRegression(Strategy):
    model = None
    
    limit_buy = 1
    limit_sell = -5

    def init(self):
        self.already_bought = False

    def next(self):
        explanatory_today = self.data.df.iloc[-1:, :]
        forecast_tomorrow = self.model.predict(explanatory_today)[0]
        ...

On the optimization function as follows:

from sklearn.tree import DecisionTreeRegressor

model_dt = DecisionTreeRegressor()
model_dt.fit(X, y)

stats_skopt, heatmap, optimize_result = bt.optimize(
    model = [model_dt],
    limit_buy=[0, 10],
    limit_sell=[-10, 0],
    maximize='Return [%]',
    method='skopt',
    max_tries=500,
    random_state=0,
    return_heatmap=True,
    return_optimization=True)

Results in the following error:

AttributeError: 'NoneType' object has no attribute 'predict'

Since I am using model=[model_dt], I expect the strategy to take the given model. Nevertheless, it takes the default model = None. Any workaround for this problem?

jsulopzs avatar Apr 26 '23 08:04 jsulopzs

maybe write model_bt out of the []

ShaharBrandman avatar Apr 30 '23 09:04 ShaharBrandman

The bug for the following minimal test case:

from backtesting import Backtest, Strategy
from backtesting.test import GOOG

class Test(Strategy):
    model = None
    
    def init(self):
        assert self.model is not None

    def next(self):
        pass
    
model = 5
bt = Backtest(GOOG, Test)
stats = bt.optimize(model = [model])

reveals itself to be in: https://github.com/kernc/backtesting.py/blob/0ce24d80b1bcb8120d95d31dc3bb351b1052a27d/backtesting/backtesting.py#L1326-L1329 where the strategy is preinitially run with no parameters.

For a workaround, set class variable: model = SomeDefaultSklApiModel().

I welcome a PR that checks maximize= string validity in some other way! 😅

kernc avatar May 03 '23 15:05 kernc

Hi there, I also get this issue, whether I have defined the class variable values or not - it also seems to be somewhat random i.e. some class variables optimize fine, others create the error below - have I misunderstood the workaround you mention above?

 if maximize not in stats: 
     raise ValueError('`maximize`, if str, must match a key in pd.Series ' 
                      'result of backtest.run()') 

CadeHalcyon avatar May 11 '23 19:05 CadeHalcyon

@CadeHalcyon Can you show some of your code? The error:

ValueError('`maximize`, if str, must match a key in pd.Series ' 
           'result of backtest.run()')

is actually quite irrelevant to this issue.

kernc avatar May 12 '23 13:05 kernc

@kernc Thanks , but I have now found the issue, I realise for some optimization formulas for the variables I had written: maximize='Return[%]'

instead of:

maximize='Return [%]'

Apols for the false alarm!

CadeHalcyon avatar May 12 '23 22:05 CadeHalcyon