ArcticDB
ArcticDB copied to clipboard
iterate_type with prefix can return incomplete results with S3 storage
https://github.com/man-group/ArcticDB/blob/master/cpp/arcticdb/storage/s3/s3_storage-inl.hpp#L359-L360
Underlying cause is that the index type (which comes before the symbol name in the S3 path) is hard-coded based on whether the keytype is ref or atom. This is incorrect, for example, with symbol list keys, where __add__ and __delete__ keys have "Unknown" index type, but compacted keys have "Timestamp".
Short term - should confirm that no current usages of iterate_type are returning incomplete sets of keys.
Longer term - fix to check all possible index types for the provided key type.
I don't think anything should have "Unknown" index type, it seems like that's the bug
At least for the purposes of iterate_type, it's assumed that all ref keys have unknown index type