spicedb
spicedb copied to clipboard
authzed
Open Question: feasibility of datastore using dynamodb
I'm looking for your thoughts on implementating a database backend for authzed in dynamodb. Zenzibar paper did specifically say that it is using spanner, but it reads like it is really zookie that plays the important role of guaranteeing causal acl. I don't have any concrete code right now, but my feeling is that it may be possible to design the db with mostly eventual consistency, with small amount of strong consistency parts. The caveat would be that eval node answers may be different from each other, abit only for a short period after write. This should be alright in real life use cases. Most IAM from big cloud providers are not strongly consistent.
Is there any technical difficulty I have overlooked?
@imacks The biggest open question is handling of zedtokens and "full consistency": if a zedtoken is provided, or full consistency is required, SpiceDB is required to honor that request by the client. Thus, there must either be a way in DynamoDB to request the fully consistent updates that have been applied or SpiceDB will have to wait until it can guarantee that the updates have been applied.
So long as this is doable (or it is acceptable to wait), I would imagine implementation should be fairly straightforward
Where strong consistency is required, we can use conditional writes and strongly consistent reads. Having said that, the biggest advantage I hope from DynamoDB backend is better latency as things scale up, which suggests an eventual consistency model.
Yeah, strong consistency will only be required when requested by the client. If the ZedToken's requirements are already met by the eventual consistency at the time of the call, or if minimize_latency is requested (which is the default), then eventual consistency is fine
Tks for the tip. I will try my hands on making a POC and then circle back. Cheers!
@josephschorr Apologies for the slow response. Day job busy busy yada yada (@~@) ...
Progress report: (1) dynamodb datastore prototype is mostly working, but still lots of unit tests missing. I suspect there is also room for improvement, e.g. by avoiding transactions in some cases. (2) I'm also working on a cassandra datastore that is looking promising performance-wise. For Cassandra, adhering to the DataStore interface has proved to be challenging. I will update you on this point in a separate issue once I feel ready to commit.
@imacks Would you be willing to share your progress? I'm quite curious :)
I just wanted to follow up on this thread as we've seen requests for DynamoDB usage in multi-region SpiceDB deployments.
DynamoDB can be used to implement a datastore for single-region deployment, but in a multi-region DynamoDB configuration, writes across regions are replicated in an eventually consistent, asynchronous fashion. This means requests with a full_consistency ZedToken in this configuration would produce incorrect results.
@imacks Checking in again to see if you have anything you'd mind sharing :)
I have quite a bit of experience with Dynamodb and Go. Also we have written a go lib around making working with Dynamo much easier. We were thinking to make it open source. @imacks happy to help with anything you need.
Edit: @imacks did you implemented the single table design? Also unable to find any feature branch around this.
