delta-rs
delta-rs copied to clipboard
delta-io
CI integration test fails randomly: test_restore_by_datetime
One of our tests (test_restore_by_datetime) in the CI - integration tests sometimes randomly fails: https://github.com/delta-io/delta-rs/actions/runs/7044053111/job/19170981661?pr=1924
There are 3 writes to the table in that test, could all 3 be within the same millisecond? The test is picking up v1 timestamp then retrieving a version for it. The error can be thrown if the snapshot version also has the same timestamp
Interesting results after some extra tracing within this test
All the history timestamps are [1703869558237, 1703869559278, 1703869560403, 1703869561411] showing that they are in sequence with 1 sec gap between them so I doubt the issue can be fixed by adding a gap between writes so I am going to remove the current delay between writes from the test
Test is looking for a timestamp 1703869559278, which is a correct value at index 1. I confirmed that is gets converted to a correct DateTime (converting it to timestamps gives a correct value: 1703869559278)
So looks like there is something wrong with the lookup logic
Problem located - this is not a flaky team IMO but a bug.
It starts here with creating a new DeltaTable with no history/timestamps loaded: https://github.com/delta-io/delta-rs/blob/02e26e5ecd7a8a64609bd21e8cb30e9bad4cc17b/crates/deltalake-core/src/operations/restore.rs#L142C44-L142C44
The consequences surface here: https://github.com/delta-io/delta-rs/blob/02e26e5ecd7a8a64609bd21e8cb30e9bad4cc17b/crates/deltalake-core/src/table/mod.rs#L884C28-L884C49 in turn leading to https://github.com/delta-io/delta-rs/blob/02e26e5ecd7a8a64609bd21e8cb30e9bad4cc17b/crates/deltalake-core/src/table/mod.rs#L643
Consequences: If your writes are fast enough to millisecond between the start of write and commit to storage - you are good, otherwise not and the restore to DataTime will not works as expected, possibly resulting in an earlier version being restored.
I'll submit a fix in the next days
