Non-monotonically increasing timestamp in spread data

[yuntai]

I have non-monotonically increasing timestamps in a spread data series which causes the spread report to fail. https://github.com/robcarver17/pysystemtrade/blob/develop/sysproduction/reporting/data/costs.py#L273

EUROSTX 2024-04-16 23:58:51.804114 ┆ 1.0 2024-04-17 10:10:24.303295 ┆ 1.0 2024-04-16 23:58:51.804114 ┆ 1.0 2024-04-17 10:10:24.303295 ┆ 1.0 2024-04-16 23:58:51.804114 ┆ 1.0

The inversion seems to be local, so that I could make a quick fix by replacing spreads_in_period = raw_spreads[start_date:end_date] with raw_spreads[(raw_spreads.index>=start_date)&(raw_spreads.index<=date_time)] without affecting the result.

But I'd like to know if you have had this problem before and what you recommend doing. Should data be filtered during data ingestion?