node-ottoman icon indicating copy to clipboard operation
node-ottoman copied to clipboard

Investigation - Refdoc Index

Open AV25242 opened this issue 4 years ago • 2 comments

Does refDoc works against object or array type of field?

If so can you come up with some examples.

AV25242 avatar Jun 15 '21 21:06 AV25242

Short answer: No.

Deep dive:

Theory: Keep in mind refdoc indexes main goals:

  • Provide a way to ensure unique values for a field (feature don't provide by Couchbase server).
  • Speed up queries by retrieving a document using only key/value operations.

To retrieve a document using only key/value operations Ottoman create some sort of a pipeline to lookup for the document, see the example below:

Technical:

  1. define a refdoc index UserSchema.index.findRefName = { by: 'name', type: 'refdoc' };

  2. when saving/updating a document, refdoc index logic will create an extra document with this structure: example of the saved document {name: 'Jhon'}

extra document created: { "User::name$Jhon" : "Jhon-document-key"}

  1. using the refdoc index function "findRefName" (defined in the UserSchema)

User.findRefName('Jhon');

Under the hood Ottoman do: build refdoc key = "User::name$Jhon" retrieve using the key the document = { "User::name$Jhon" : "Jhon-document-key"} collection.get('User::name$Jhon') retrieve the real document by using the refdoc document value as key = "Jhon-document-key" collection.get('Jhon-document-key')

Lineal graph: User.findRefName('Jhon') -> collection.get('User::name$Jhon') -> collection.get('Jhon-document-key')

gsi-alejandro avatar Jul 09 '21 18:07 gsi-alejandro

Maybe we could try to add support for arrays and objects but I don't recommend it. Due to the key size limitation, even now with only support for scalar values mainly strings we had to implement validation for key size too long.

Unable to store refdoc index for ${ref}, Maximum key size is 250 bytes

gsi-alejandro avatar Jul 09 '21 18:07 gsi-alejandro