materialize icon indicating copy to clipboard operation
materialize copied to clipboard

Expose reclocking information as a relation in Materialize

Open petrosagg opened this issue 3 years ago • 1 comments

The storage controller currently exposes collections identified by a GlobalId. These collections correspond 1-1 to a TVC that one can read by demonstrating its id.

We would like to change this relationship so that each storage collection correspond to two parts (maybe more in the future), its data and its timestamp binding TVCs. Users of storage will now need to specify which part of a storage collection they need by specifying an (id, type) tuple.

Implementing this depends storing the timestamp bindings in a persist collection (#11907) so that readers can read the data.

petrosagg avatar Apr 19 '22 19:04 petrosagg

And there's a design nearing completion in #11718!

benesch avatar Apr 20 '22 04:04 benesch

Note that before we do this we want to migrate this data to the more Timely-friendly format which Petros is already doing in memory

sploiselle avatar Nov 16 '22 19:11 sploiselle

Implemented as #15813

aljoscha avatar Feb 23 '23 12:02 aljoscha