ArcticDB
ArcticDB copied to clipboard
Persistent storage tests: Run against multiple baselines, normalize, and extend test coverage
PR #1462 made a start on extending persistent storages test coverage.
But the existing persistent storage tests run on a large matrix, checking interop between every 2 elements in this Cartesian product:
{reader, writer}
X
{windows, linux}
X
{python3.6..3.11}
X
{master ArcticDB, single baseline ArcticDB}
They do not support testing where one version writes, and then another version tries to modify that data.
To support tests like that, eg the tests in PR #1462 , we should:
- Make the tests in the Cartesian product as simple as possible (basic read + write)
- Introduce a new sort of persistent testing that runs just on Linux with Python3.11
- On this new setup we can write more involved tests that can freely mutate data (as there won't be another test run also trying to mutate it)
- We can introduce more testing without massively slow test runs
- We can run against an arbitrary number of baselines (eg 1.6.x, 3.0.0, 4.0.x)
We will prioritise this after we have a dedicated test engineer