bonsaidb icon indicating copy to clipboard operation
bonsaidb copied to clipboard

Add support for document dependencies (foreign keys)

Open ecton opened this issue 3 years ago • 2 comments

One of the limitations of the commerce benchmark (#131) is that foreign keys aren't being evaluated in BonsaiDb. We should be able to add support for these sorts of dependencies by creating a mechanism to relate two collections together.

On-disk, this will be implemented as two Nebari trees, both containing simple mappings of (CollectionName, u64) pairs between both collections, allowing fast lookups in either direction.

When inserting or updating a document, this new trait should be asked to supply any relationships the document contains. The indexes must be updated during apply_transaction.

When a document is about to be deleted, a check is performed to ensure there are no dependencies through any relationships.

When a document is deleted, the deleted document's entry is removed from both indexes.

ecton avatar Jan 15 '22 00:01 ecton

Another thought occurred to me while working on #238 -- it would be nice to be able to make dependencies on keys of unique views as well.

ecton avatar Apr 05 '22 16:04 ecton

On Discord a question was asked about potential API design for this feature. At its core, I think the Collection trait would need a new method that returns a collection of a new DocumentDependency type (naming suggestions are always welcome):

trait Collection {
  // ...
  fn dependencies(document: &BorrowedDocument<'_>) -> Vec<DocumentDependency> {
    Vec::new()
  }
}

enum DocumentDependency {
  Document {
    collection: CollectionName,
    id: DocumentId,
  },
  View {
    view: ViewName,
    key: Bytes
  },
}

It may make more sense to use OwnedDocument for the dependencies method, I wouldn't know for sure until looking at how it would be implemented more closely.

What I'm not sure of is how one might create a good API where dependencies could already be a CollectionDocument<T>. Additionally, I'm not quite sure what the best macro syntax might be to allow defining dependencies within the #[derive(Collection)] macro.

It may make more sense to create a new type called DocumentDependencies that provides a builder-style pattern to make it more ergonomic to implement dependencies().

ecton avatar Dec 08 '22 17:12 ecton