Expose FoundationDB versionstamp in API

[richardartoul]

Is your feature request related to a problem? Please describe. Databases that make sequencing simple and easy are in short supply, I can really only think of FoundationDB. It would be nice to add Tigris to the list :)

NOLA leverages this FDB functionality to ensure linearizability of actor invocations: https://github.com/richardartoul/nola/tree/master/proofs/stateright/activation-cache

Setting the commit version in the key/value will also be useful for implementing systems where ordering matters (like a journaling or FIFO queing system for example).

Describe the solution you'd like I'd like to be able to read the FDB transaction's readversion in my transaction and provide a placeholder value in my primary key and document values that will be filled in with the commit version at commit time. Specifically the Tigris transaction object should expose a GetReadVersion similar to this method.

In addition, the logical equivalent of SetVersionStampedKey() and SetVersionStampedValue() should also be exposed.

I'm not exactly sure what the API should look like, but since the primary key drives the FDB key in tigris, someway to encode the primary ID in such a way that the commit version will be encoded it in a way that makes sense for sorting. I.E I should be able to insert documents into a collection in such a way that they end up ordered by commit order. Ideally this allows the userVersion its to be set as well like this: https://pkg.go.dev/github.com/apple/foundationdb/bindings/[email protected]/src/fdb/tuple#IncompleteVersionstamp and this: https://pkg.go.dev/github.com/apple/foundationdb/bindings/[email protected]/src/fdb/tuple#Tuple.PackWithVersionstamp

For the value, I think just some way to encode it as a "value" in the document should suffice.

Describe alternatives you've considered I can't think of any way to get commit ordering.

[garrensmith]

That makes sense could this be solved via the secondary index functionality? We currently have _tigris_created_at and _tigris_updated_at we could allow a user to specify that they want a sequence index as well _tigris_seq that we then use a version stamp to order the documents. Something like (versionstamp, primary_key)

That would give you the ordering you the commit order. I'm not sure on exposing the SetVersionStampedKey and SetVersionStampedValue. I can't quite think of a good API for that and if that would be needed if we can provide the sequence index. I think the record layer does something similar https://github.com/FoundationDB/fdb-record-layer/blob/main/docs/Overview.md#indexing-by-version

[himank]

Or we can expose versionstamp as one of the primary key alternatives? Similar to what you have described we can call it "sequencer/commitOrder" and then if this is set in the schema then we fill it SetVersionStampedKey and store it in user's payload as well.

[efirs]

Along with wall clock time type we can introduce cluster time field type.

So user can define it in the models like this:

type User struct {
  CreateTime tigris.Time `tigris:"created_at"`
  UpdateTime tigris.Time `tigris:"updated_at"`
  WallTime time.Time `tigris:"updated_at"`
}

It translates to the following in the schema:

...
"UpdateTime" : {
  "type" : "string"
  "format" : "cluster-time"
  "updated_at" : true
}
...

The updated_at and created_at annotations behave the same as for wall time.

On the lower level we persist it at the known offset in the value, using SetVersionStampedValue, for example right after our internal.TableData wrapper. Also if the field is annotated as primary key we set it in the key using SetVersionStampedKey too. On read we populate values in the user payload from those two known locations.

For the read version we can return it as a part of begin transaction response, so it's available to the user during the transaction:

u := db.GetCollection[User](ctx)

_ = u.Insert(ctx, &User{}) // this auto fills the times

db.Tx(ctx, func(ctx context.Context) {
 thisTxStartTime := db.Time(ctx) // available inside transaction

 u1, _ := u.ReadOne(ctx)

 fmt.Printf("logical time of row creation: %v\n", u1.CreateTime)
 fmt.Printf("logical time of row update: %v\n", u1.UpdateTime)
})

[himank]

@efirs One correction to your suggestion, SetVersionStampedValue doesn't allow setting values from outside. It is also something we don't want to expose as we use it for atomic metadata management. Also, if SetVersionStampedKey is not annotated on primary key then we need an extra write.