Templated index fields not updated

[shishkin]

Describe the bug

I use GSI templated fields like gs2sk: { type: String, value: "${status}#${appointment}" } and expect them to be updated whenever I update the item's actual fields like status and appointment. However, that is not happening.

To Reproduce

Steps to reproduce the behavior:

See: https://github.com/shishkin/dynamodb-onetable/blob/72e19196fc856a208947eb72b2a7a15fe408cfdb/test/debug.ts

Model:

    User: {
      pk: { type: String, value: "${_type}" },
      sk: { type: String, value: "${id}" },

      gs1pk: { type: String, value: "${_type}" },
      gs1sk: { type: String, value: "${status}#${appointment}" },

      id: { type: String, required: true },
      status: { type: String, required: true },
      appointment: { type: Date, required: true },
    },

Test:

  await users.create(user, { exists: null });
  await users.update({ id: user.id, status: "CANCELLED" });
  const u1 = await users.get({ id: user.id }, { hidden: true });
  expect(u1?.gs1sk).toMatch(/^CANCELLED#/);

Note that gs1sk is comprised of status and appointment, but update only provides new status.

OneTable fails to update gs1sk field:

'OneTable result for "update" "User"',
      {
        cmd: {
          ConditionExpression: '(attribute_exists(#_0)) and (attribute_exists(#_1))',
          ExpressionAttributeNames: {
            '#_0': 'pk',
            '#_1': 'sk',
            '#_2': 'gs1pk',
            '#_3': 'id',
            '#_4': 'status',
            '#_5': 'updated'
          },
          ExpressionAttributeValues: {
            ':_0': { S: 'User' },
            ':_1': { S: '123' },
            ':_2': { S: 'CANCELLED' },
            ':_3': { S: '2022-07-08T15:00:18.458Z' }
          },
          TableName: 'DebugTable',
          ReturnValues: 'ALL_NEW',
          UpdateExpression: 'set #_2 = :_0, #_3 = :_1, #_4 = :_2, #_5 = :_3',
          Key: { pk: { S: 'User' }, sk: { S: '123' } }
        },
        items: [
          {
            id: '123',
            status: 'CANCELLED',
            appointment: 2022-07-08T15:00:18.355Z,
            created: 2022-07-08T15:00:18.355Z,
            updated: 2022-07-08T15:00:18.458Z
          }
        ],
        op: 'update',
        properties: {
          pk: 'User',
          sk: '123',
          gs1pk: 'User',
          id: '123',
          status: 'CANCELLED',
          _type: 'User',
          updated: '2022-07-08T15:00:18.458Z'
        },
        params: { exists: true, parse: true, high: true, checked: true }
      }

Of course, if I add appointment value to the update command, all works as expected. But in my case, I don't have the original item appointment when I update status to CANCELLED.

Expected behavior

I understand why it doesn't update, but maybe OneTable could assign the missing value from the item itself (issue a more elaborate DynamoDB update statement) or just give an error disallowing update without all index fields being consistent.

[mobsense]

Thank you for the excellent issue description. You make it easy to quickly respond. :-)

Yes, update cannot update a field that uses a value template for which all the component fields are not supplied.

The reason is that otherwise, update would need to incur a read to fetch the full prior item to get the missing field values. So OneTable cannot update such partial value templates.

We could warn when updating an item and supplying partial value template fields. But there are probably use cases where this is exactly what the user wants. Let us mark this as enhancement and think this over.

[dev-embedthis]

We've dug into this and the warning would trigger a lot of false positives. Sorry.