new approach for ob and bos_choch, addition of shl_forward

[CosmicTrader]

Hello @joshyattridge and other fellow developers, I have added much awaited faster implemantation of order block in this pull request.

While working with the library, I found few issues in the output and speed of few functions comparing to tradingview indicator smartmoneyconcepts by LuxAlgo which are as below :

1. swing high and lows are different in tradingview, they use only forward candles when deciding if a high or low is swing high or low for given swing_length. So I have added a new function swing_highs_lows_forward which addresses this issue.

2. BoS and CHoCH in the code does not provide us with all the values, only few instances were coming in output compared to all the swing_high_low and BrokenIndex column for many of them was missing which I have fixed in this one and I have added one extra column for BrokenDate which gives us the datetime when the structure was confirmed.

3. ob function was drastically slow and gave only active order blocks, which is not useful for backtesting large data. I have tried to increase that speed (very much faster now then previos, but still some room for improvement). Now we get all the order blocks even those which are mitigated so its lot easier to backtest them. I have added 2 new columns

• ConfirmDate : datetime when structure responsible for forming order block was confirmed
• MitigationDate : datetime when order block was mitigated

So the order block with mitigation date is inactive and vice versa.

I couldn't understand how OBVolume was calculated so that calculations and percentage values are not added. I will implement that as well, once its calculations are clearified.

Thank you for this awesome library and efforts in building and maintaining it.

[CosmicTrader]

@rafalsza If the code that you provided gives accurate results and it is also faster for large datasets, then I would recommend to use it. But if you are testing for 1 minute data of past 10years on forex pair, I would prefer little bit complexity for speed over a simple implementation. and since your code uses iterating over the whole dataframe, there is no chance it would be faster than my implementation. If there is no issue of speed, I would really appriciate your take of simplicity.

[rafalsza]

@rafalsza If the code that you provided gives accurate results and it is also faster for large datasets, then I would recommend to use it. But if you are testing for 1 minute data of past 10years on forex pair, I would prefer little bit complexity for speed over a simple implementation. and since your code uses iterating over the whole dataframe, there is no chance it would be faster than my implementation. If there is no issue of speed, I would really appriciate your take of simplicity.

1)times are the same 2024-06-21 13:42:59.961 | DEBUG | main::157 - volumized_ob processing time (for ohlc length=2501): 10.21 seconds 2) calculation is wrong correct: your approach:

[CosmicTrader]

can you please provide me the ohlc data and output data for the same. [email protected]

[rafalsza]

can you please provide me the ohlc data and output data for the same. [email protected]

btc_1d.csv ob_data.csv

[Asite06]

Dear @CosmicTrader , @rafalsza thanks for great work. If you are using this lib for drawing charts it is perfect but if you are up to algo trading entire lib has lookahead bias. I couldnt solve issues and gave up just saying.

[joshyattridge]

@Asite06 please can you explain what you mean by this lib having a look ahead bias, how are you using this lib as you should be calling the functions on each new candle passing the past X amount of candles to the function so I can't see how there is a look ahead bias.

[Asite06]

Dear @joshyattridge i dont know if you ever used Freqtrade bot. According to this bot while backtesting it gives entire df at once so you cant use negative iloc, shift and max min without window len. For example;

    swing_highs = future_highs == future_highs.rolling(
        window=swing_length + 1, min_periods=swing_length
    ).max().shift(-swing_length)

    swing_lows = future_lows == future_lows.rolling(
        window=swing_length + 1, min_periods=swing_length
    ).min().shift(-swing_length)

Causes bias because of negative shift. I have tested so many implamention by me even i have solve some of them i couldnt fix all. My purpose is not to blame you i am gratefull but if this library is for algo trade we must fix it. İf it is freqtrades issue please advice me better one. Thanks

[joshyattridge]

@CosmicTrader, Thank you for your contribution, can you play make sure all the unit tests pass. Then I will take a look at the changes and work towards merging these additions.

[CosmicTrader]

@CosmicTrader, Thank you for your contribution, can you play make sure all the unit tests pass. Then I will take a look at the changes and work towards merging these additions.

Hello @joshyattridge, sorry couldnt complete this before, I was working on other projects. In the tests, I dont know why it is failing even after changing the test script to convert datetime columns type. FYI, all the order blocks and order blocks are present in the output even if it is broken.

[jjphung]

@Asite06 please can you explain what you mean by this lib having a look ahead bias, how are you using this lib as you should be calling the functions on each new candle passing the past X amount of candles to the function so I can't see how there is a look ahead bias.

Hi, I would like to chime in here to speak about this and maybe this is a misunderstanding on my end too. Let's look at Fair Value Gap (FVG) function:

• Uses shift(1) (previous candle) and shift(-1) (next candle)
• Has look-ahead bias since it requires the next candle's data to calculate the current value

If my understanding here is incorrect, please help me understand. Thank you!