recommendation for handling optional attributes (in keys or elsewhere)

[silberistgold]

Hi there,

thanks a lot for the release of v1.15.0 which resolves all blockers for me, so I can finally upgrade (currently using 1.10.2). 🎉

While upgrading I encountered a couple of issues related to the handling of optional attributes and I am not sure how to solve them:

1. optional attributes I have an optional attribute on my Entity (not part of a key) declared like device?: string;. When I try to update an object and set the value of object to undefined (e.g. with an update body like {device: undefined} ) I get the following error: Invalid UpdateExpression: An expression attribute value used in expression is not defined; attribute value: :UE_device. It seems typedorm filters out undefined values and the generated dynamo expression is missing this value. Is this a bug or is it intended behaviour? What is the recommended way to handle this case? Extending the type definition to device?: string | null; and updating with {device: null} works, but might not be desired in all cases.

2. nullable attributes as part of key When using an attribute with type device?: string | null; in a key, I get the following error: Error: "device" is used in key XYZ, thus it's type must be or scalar type, if attribute type is Enum, please set "isEnum" to true in attribute decorator. I can remove the null but then I am not able to correctly set the attribute to be undefined. I read about the isSparse option, which is supposed to be released in v1.16.0 but seems to be already included in v1.15.0, but enabling this on the index does not seem to change the behaviour.

It might also be a good idea to have some examples with optional values (and sparse indices) in the readme as I guess this is a very common case.

[whimzyLive]

Thanks for reporting this @silberistgold. Looks like a bug to me, I'll have a look.

Ideally I would expect the TypeDORM to auto remove the attribute from the attribute update expression when the value is set to undefined and when it is of type null it should let it go as is mainly because there could be reasons where someone would be wanting to null out a value for an attribute in DynamoDB.

[joshstrange]

I'm running into this issue as well I'm not sure what the "blessed" approach is.

I'd prefer to do

@Attribute({
  default: null // <- Not allowed currently, so I set it in my data access layer that calls TypeDORM
})
myOptionalProperty: string | null;

Along with a sparse index (isSparse) but that's not working, I get the following error:

Error: "myOptionalProperty" is used in key MYPREFIX#{{myOptionalProperty}}, thus it's type must be or scalar type, if attribute type is Enum, please set "isEnum" to true in attribute decorator.

However if I change that to

@Attribute()
myOptionalProperty?: string;

And I don't define it (either set it to undefined or don't touch it at all) I get the following error:

{
  "name": "SparseIndexParseError",
  "message": "Failed to parse \"MYPREFIX#{{myOptionalProperty}}\""
}

My index looks like this:

GSI2: {
  partitionKey: 'MYPREFIX#{{myOptionalProperty}}',
  sortKey: 'MYPREFIX2{{myOptionalProperty2}}', // I'm using another optional property here 
  type: INDEX_TYPE.GSI,
  isSparse: true // <- This should be all I need right?
},

[GTRanger]

This also seems to be a problem if you specify a key but assign the value as undefined. For example:

const userId = 1
const email = "[email protected]"
const nickname = undefined
entityManger.update(
  User, 
  {userId: userId}, 
  {
    email: email, 
    nickname: nickname <--- This will result in an error because the key will exist, but be undefined
  }
)

Since JSON.stringify doesn't stringify undefined values, this can get missed when logging the update expression to console, but if you use the following logic (for example I added this to the transactWrite function document-client-v3.js):

const replacer = (key, value) => value === undefined ? null : value
console.log(JSON.stringify(input, replacer))

This will result in the following update expression:

{
    "Update": {
        "TableName": "Users",
        "Key": {
            "PK": "1",
            "SK": "info"
        },
        "ReturnConsumedCapacity": null,
        "UpdateExpression": "SET #UE_email= :UE_email, #UE_nickname = :UE_nickname",
        "ExpressionAttributeNames": {
            "#UE_email": "email",
            "#UE_nickname": "nickname",
        },
        "ExpressionAttributeValues": {
            ":UE_email": "[email protected]",
            ":UE_nickname": null, <--- THIS IS ACTUALLY UNDEFINED AND CAUSES AN ERROR
        }
    }
}

Note that this problem is entirely avoided if you never include a key with the undefined value in the update body:

const userId = 1
const email = "[email protected]"
const nickname = undefined
entityManger.update(
  User, 
  {userId: userId}, 
  {
    ...(email && {email: email}), 
    ...(nickname && {nickname: nickname}),  <--- does not cause an error since the undefined key will not be spread
  }
)