darts icon indicating copy to clipboard operation
darts copied to clipboard

Feature/historical retrain on condition

Open FBruzzesi opened this issue 3 years ago • 1 comments

Addresses #135 and #623

Summary

Implemented support for new types in retrain argument of historical_forecasts method. Now it accepts bool, (positive) int and Callable (returning a bool) data types.

The new behaviour is as follows (copying from the updated docstring):

  • In the case of bool: retrain the model at each step (True), or never retrains the model (False). Not all models support setting retrain to False. Notably, this is supported by neural networks based models.
  • In the case of int: the model is retrained every retrain iterations.
  • In the case of Callable: the model is retrained whenever callable returns True. Notice that the arguments passed to the callable are as follows:
    • pred_time (pd.Timestamp): next timestamp to predict (retraining happens before)
    • series (TimeSeries): train series up to pred_time
    • past_covariates (TimeSeries): past_covariates series up to pred_time
    • future_covariates (TimeSeries): future_covariates series up to pred_time + series.freq * forecast_horizon

Other Information

  1. In order to achive this behaviour, I also added a decorator function, called _retrain_wrapper, in utils/utils.py, which passes to the wrapped function only the original signature arguments, and raises a ValueError in the case that the provided function doesn't return a boolean value.
  2. To test the behaviour, I wrote a test, called test_backtest_retrain, in test_local_forecasting_models.py as suggested in #623

FBruzzesi avatar Aug 10 '22 12:08 FBruzzesi

Codecov Report

Base: 93.61% // Head: 93.61% // Increases project coverage by +0.00% :tada:

Coverage data is based on head (bd6a92e) compared to base (4a27edd). Patch coverage: 88.23% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1139   +/-   ##
=======================================
  Coverage   93.61%   93.61%           
=======================================
  Files          81       81           
  Lines        8328     8330    +2     
=======================================
+ Hits         7796     7798    +2     
  Misses        532      532           
Impacted Files Coverage Δ
darts/utils/__init__.py 100.00% <ø> (ø)
darts/models/forecasting/forecasting_model.py 96.49% <87.50%> (+0.39%) :arrow_up:
darts/utils/utils.py 93.51% <88.88%> (-0.43%) :arrow_down:
darts/timeseries.py 92.23% <0.00%> (-0.07%) :arrow_down:
...arts/models/forecasting/torch_forecasting_model.py 87.45% <0.00%> (-0.05%) :arrow_down:
darts/models/forecasting/block_rnn_model.py 98.24% <0.00%> (-0.04%) :arrow_down:
darts/models/forecasting/nhits.py 98.55% <0.00%> (-0.02%) :arrow_down:
darts/datasets/__init__.py 100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov-commenter avatar Aug 11 '22 20:08 codecov-commenter