[KV Feedback]: Atomic check greater or equal versionstamp

[vwkd]

🔍

• [x] Did you search for existing issues?

Type of feedback

Feature request

Description

Currently, atomic checks test for strict equality of the versionstamp. There should be an option to loosen the check to greater or equal.

This is useful for a CRUD API that stores each piece of data of a cohesive collection individually and needs to ensure atomic consistency.

The API might not be able to dump the whole JSON object at a key, for example if it models relationships between collections which can be individually read/set/updated/deleted, or if it wants to selectively read/update individual pieces of a collection. It can store each piece of data at an individual key and assemble the collection for the user. However, this makes it currently impossible to ensure atomic consistency across API calls, as there is no single versionstamp for the whole collection that can be returned to the user, as each piece of data has an individual potentially different versionstamp.

With this option, the API could return the greatest versionstamp of each of the pieces as versionstamp for the whole collection to the user. In a subsequent mutation call, the API can check that the user-provided versionstamp for the collection is greater than or equal to the internal versionstamp of each of its pieces. If the check passes, no piece of the collection was updated in the meantime since no versionstamp is larger, and the pieces can be safely mutated.

Possibly related https://github.com/denoland/deploy_feedback/issues/408. There's no versionstamp that can be returned for an assembled book object containing its author. The versionstamp from db.atomic().check(book).check(author).commit() can't be used either to represent the collection of book and author.

Choosing only the largest versionstamp loses information of the other smaller versionstamps, but this is fine since we already verified their consistency until the time of the largest versionstamp. A non-information-losing versionstamp would need to be some aggregate of its constituent versionstamps, like the multiple of prime factors and the check tests whether it's divisible by a factor, which would probably be more complex and less efficient.

Steps to reproduce (if applicable)

A minimal example for a collection of a book and its author.

Assume the book and author are inserted/updated separately, resulting in potentially different versionstamps.

What versionstamp should the assembled book with its author have?

const db = await Deno.openKv(":memory:");

/**
 * Some sample data
 */
await db.set(["authors", 11], { name: "William Writer" });
await db.set(["books", 1], { title: "Tale of Tales", author: 11 });

/**
 * Read book and its author
 */
async function readBook(id) {
  let authorRes;
  let bookRes;
  let check = { ok: false };
  
  while (!check.ok) {
    bookRes = await db.get(["books", id]);
  
    const authorId = bookRes.value.author;
    
    authorRes = await db.get(["authors", authorId]);
  
    check = await db
      .atomic()
      .check(authorRes)
      .check(bookRes)
      .commit();
  }
  
  if (!bookRes.value) {
    return { id, value: null, versionstamp: null };
  }

  const value = {
    ...bookRes.value,
    author: authorRes.value,
  };
  
  // todo: what to do here? no versionstamp for both
  const versionstamp = "fooooo";
  
  return { id, value, versionstamp };
}

/**
 * Delete book and its author
 */
async function deleteBook(id, versionstamp) {
  const bookKey =  ["books", id];

  const bookRes = await db.get(bookKey);

  const authorId = bookRes.value.author;
  
  const authorKey = ["authors", authorId];

  return db
    .atomic()
    // todo: what to do here? no versionstamp for both
    .check({ key: bookKey, versionstamp })
    .check({ key: authorKey, versionstamp })
    .delete(bookKey)
    .delete(authorKey)
    .commit();
}

Expected behavior (if applicable)

No response

Possible solution (if applicable)

An option to losen the atomic check to greater than or equal to versionstamp.

    // ...
    const versionstamp = [bookRes.versionstamp, authorRes.versionstamp].sort()[0];
    // ...

    // ...
    .check({ key: bookKey, versionstamp }, { loose: true })
    .check({ key: authorKey, versionstamp }, { loose: true })
    // ...

Additional context

No response

[vwkd]

One workaround is to designate one of the pieces of data as "primary" and use its versionstamp for the collection.

For example, we could use the versionstamp of the id field of a book as the versionstamp for the assembled book.

Now if we mutate any of the other fields, we need to mutate (without changing, just re-setting) the id field in the same atomic transaction, such that its versionstamp remains equal to the largest.

Of course it's not ideal, as we now need to remember to also re-set the id field each time we mutate any field, which is fragile.