roll operator

[mccanne]

As a first-step toward a unique primary-key, updatable Zed data store, we will add a manual command (tentatively) called roll to roll forward a sequence of primary-key updates into its latest state.

This will allow one to implement manually-computed updates by ingesting data that conforms to this model.

In accordance with our command-first architectural philosophy, this command will serve as a baby step toward primary-key pools that have DML-like capabilities, which we will implement later after gaining experience with the roll operator.

For now, we will keep it simple where roll will take one argument, the primary key id, e.g.,

roll id 

Roll will then presume there are these two additional fields in each value:

• version - an increasing version number of each update so they can be sorted in their proper order
• op - an operator which is one of "c" (create), "u" (update), "d" (delete)

The input to roll is assumed to be sorted by primary key and secondarily by the version. This will allow for an efficient implementation that can operate upon and release one value at a time. When we have secondary pool keys, the pool can be presorted so no runtime sort will be needed.

For creates, there is a data field, which holds the initial value. It is legal to create values on top of previously defined value, in which case, the new value replace the old value in its entirety.

For updates, there is a flattened array of path/value pairs in a field called updates. If a field doesn't exist it is created. An update is equivalent in semantics to a record spread on the existing data field.

{...data,field1:val1,field2:val2,...}

The output of roll is a set of create records of the form:

{id:...,version:...,op:"c",data:{...}}

Note that this pattern

id==10 or id==11 | roll id | yield data 

would give the two values for id's 10 and 11 without the metadata.

For deletes, there is no data or updates field. A delete for a given id is analogous to a tombstone in other systems.