last traded price lacks some info due to the logic of the parser

[BenoitLondon]

in the betfair BASIC files if some horse has not been traded at some timepoint, it doesn't appear in the json line for that timepoint

whereas in your code IIUC it is rolled forward to every timepoint. It 's actually correct as it is still the last traded price(!) but we lose the information when it was actually last traded at that price

e.g market file "1.212040273.bz2"

horse id 41133899 the last traded price 2.94 is rolled over in the next update

with betfair_data ('1.212040273', 'To Be Placed', 'PLACE', '2023-04-01 01:25:00', '2023-04-01 01:20:21.962000', 278.038, False, 'ACTIVE', '2. Crosshaven', 41133899, 3.05, None) ('1.212040273', 'To Be Placed', 'PLACE', '2023-04-01 01:25:00', '2023-04-01 01:21:21.961000', 218.039, False, 'ACTIVE', '2. Crosshaven', 41133899, 2.94, None) ('1.212040273', 'To Be Placed', 'PLACE', '2023-04-01 01:25:00', '2023-04-01 01:22:22.001000', 157.999, False, 'ACTIVE', '2. Crosshaven', 41133899, 2.94, None) ('1.212040273', 'To Be Placed', 'PLACE', '2023-04-01 01:25:00', '2023-04-01 01:23:21.968000', 98.032, False, 'ACTIVE', '2. Crosshaven', 41133899, 3.1, None)

with my R parser (much slower than yours!)

   rc.ltp    rc.id                  pt      mkt.id mc.version
1:   2.92 41133899 2023-04-01 01:15:21 1.212040273         NA
2:   2.94 41133899 2023-04-01 01:17:21 1.212040273         NA
3:   2.92 41133899 2023-04-01 01:18:22 1.212040273 5147358276
4:   3.05 41133899 2023-04-01 01:20:21 1.212040273         NA
5:   2.94 41133899 2023-04-01 01:21:21 1.212040273         NA
6:   3.10 41133899 2023-04-01 01:23:21 1.212040273         NA
7:   3.20 41133899 2023-04-01 01:24:21 1.212040273         NA
8:   3.05 41133899 2023-04-01 01:25:21 1.212040273 5147363556

you can see there is no price update at 01:22:21 as there has been no trade on this horse from 01:21:21 to 01:22:21

I can't find the logic in your code where you roll forward the last_traded_price but it would be great if you could fix that or provide sthg like last ever traded price and the real last traded price as it appears in the files with the correct timestamp So maybe one fix would be to have letp and ltp and ltp to be None when it wasnt there in the json and letp the value you roll forward?

Thank you for that cool package

PS: a wrong solution is to record only ltp changes but two equal consecutive values could be legit trades!