dwn-sdk-js icon indicating copy to clipboard operation
dwn-sdk-js copied to clipboard

Replace `messageCid` as the cursor value.

Open LiranCohen opened this issue 1 year ago • 1 comments

We currently use messageCid as the cursor value when querying for both Events and Records, however this becomes an issue if the particular message representing a messageCid gets deleted between subsequent calls.

Instead of using the messageCid as the cursor value, or some opaque curser that is only relevant to a given MessageStore implementation, we would like to create a cursor that could be represented and understood across multiple implementations.

Currently when querying we sort by a sortProperty and use the sortValue of the property as well as the itemId (messageCid) as a tie-breaker, therefore we could create a cursor that represents the sortValue and the itemId.

Steps to encode a cursor

  1. Take the itemId(messageCid) of the item which represent the cursor point.
  2. Take the sortValue of the given sortProperty and JSON.stringify() it. (ex. "2000-01-01T10:20:30.123456Z" for datePublished, true for published, or 12345 for dataSize).
  3. Join the sortValue and itemId ([sortValue, itemId].join()) by a chosen delimiter into a single string.
  4. base64URLencode this string.

When receiving a query with a cursor, the DWN/MessageStore can decode the cursor and then return results that are gt(>) the sortProperty value + itemId as a tie breaker.

Example Cursor encode/decode:

export class QueryUtility {

  public static encodeCursor(itemId: string, value: string | number | boolean): string {
    return Encoder.stringToBase64Url(this.segmentJoin(JSON.stringify(value), itemId));
  }

  public static decodeCursor(cursor: string):{ itemId: string, sortValue: string | number | boolean } {
    const decoded = Encoder.base64UrlToString(cursor);
    const [ cursorValue, itemId ] = decoded.split(this.delimiter);
    if (cursorValue === undefined || itemId === undefined) {
      throw new DwnError(
        DwnErrorCode.CursorInvalidFormat,
        `The cursor provided ${cursor}, is invalid.`,
      );
    }

    let sortValue: any;
    try {
      sortValue = JSON.parse(cursorValue);
    } catch (error) {
      // do nothing if cannot parse will fail below
    }

    switch (typeof sortValue) {
    case 'boolean':
    case 'number':
    case 'string':
      return { itemId, sortValue };
    default:
      throw new DwnError(
        DwnErrorCode.CursorInvalidValue,
        `${typeof sortValue} is not supported as a cursor value`
      );
    }
  }

  private static delimiter = `\x00`;
  private static segmentJoin(...values: string[]): string {
    return values.join(this.delimiter);
  }

}

LiranCohen avatar Dec 04 '23 23:12 LiranCohen

discussed with @thehenrytsai

Will just make it a cursor object

{
  cursor: {
    value  : boolean | string | number,
    itemId : string,
  }
}

LiranCohen avatar Dec 05 '23 00:12 LiranCohen