Look ahead bias ?

[designboxcom]

Hello, I am currenty learning the mechanics of freqtrade and its strategies and was interrested by your approach to find extrems. I am not an expert and the process of learning is seamly a long road but, stil, I try my best to understand how strats and signals work at best. Thus, I wondered if your strat doesn't suffer a so-called look-ahead bias ? The gap between backtesting and dry-run is simply too enormous. On one side, the backtesting results are simply fantastic but, on another side, running a dry-run is totally another history. Poor results and bad extrems detection.

[ivanproskuryakov]

Hi @designboxcom, the strategies in this repo started from the basic idea of finding lows and highs with the utilization of numpy's signal.argrelextrema methods.

The strategies performed well when the whole chunk of data was available - say backtesting, but required correction to filter out local minimus look-ahead bias, which you pointed out correctly.

Initially, this correction was made with simple fixed percentage drop checks, to filter out percentage drop drops of less than 5% and later re-implemented in a dynamic manner to adjust to catch a broader range of assets(Bitcoin and altcoins fluctuate currently).

Price drops check still was not catching fake deeps, with the second correction followed to look for a sequence of buying candles aimed to bypass fake deeps. This new correction relies on counting candles on RSI[7] for the given range, now the range is between 40 and 60.

Extremes search

• https://github.com/ivanproskuryakov/ta-search/blob/master/user_data/strategies/TaSearchDynamic.py#L49
• https://stackoverflow.com/a/50836425/2235085

Price drop check

• https://github.com/ivanproskuryakov/ta-search/blob/master/user_data/strategies/TaSearchDynamic30m.py#L31

Fake deep correction

• https://github.com/ivanproskuryakov/ta-search/blob/master/user_data/strategies/TaSearchDynamic30m.py#L46

[Daniel595]

Running lookahead-analysis from 20231226-20230105 says no bias detected:

2024-01-06 00:10:38,315 - freqtrade.optimize.analysis.lookahead - INFO - Found targeted trade amount = 20 signals.
2024-01-06 00:10:38,316 - freqtrade.loggers.set_log_levels - INFO - Restoring log verbosity.
2024-01-06 00:10:38,316 - freqtrade.optimize.analysis.lookahead - INFO - TaSearchLevelG15m: no bias detected
2024-01-06 00:10:38,316 - freqtrade.optimize.analysis.lookahead_helpers - INFO - Checking look ahead bias via backtests of TaSearchLevelG15m.py took 990 seconds.
| filename             | strategy          | has_bias   |   total_signals |   biased_entry_signals |   biased_exit_signals | biased_indicators   |
|----------------------+-------------------+------------+-----------------+------------------------+-----------------------+---------------------|
| TaSearchLevelG15m.py | TaSearchLevelG15m | False      |              20 |                      0 |                     0 |                     |

If this is sufficient to determine if lookahead bias is present.

[Daniel595]

But it's true, performance on dry run is very different though :-D