Upsert with nil value of the attribute present in GSI (update_attributes works)

[arekt]

Context: Table Foo has attribute "bar" (string). Sometimes I do want to reset "bar" attribute to nil.

Events:

  Foo.upsert(1, {bar: nil}) -> OK
  Add GSI with "bar" (hash) key
  Foo.upsert(1, {bar: "baz"} -> OK
  Foo.upsert(1, {bar: nil}) -> Error (related to nil value of attribute in GSI)

I was sure I can have GSI with some records without "bar" attribute, so was little bit surprised at first. In the DynamoDB console, it is possible to remove "bar" attribute, it's not possible to assign nil value to string attribute. (that's ok for me) Current workaround is to do update_attributes instead.

Workaround:

  foo = Foo.find(1) || Foo.new id: 1
  foo.update_attributes({bar: nil}) -> OK (it actually removes bar attribute and there's no error about GSI)

Summary: It feels like a bug but because of the workaround it's not urgent.

[andrykonchin]

Good catch 👍 Thanks for reporting this issue.

Looks like it's an internal limitation of DynamoDB. Attribute that is used as GSO key couldn't be NULL or empty string (but an attribute can be just not present in an item). And it contradicts some statement in the documentation:

Avoid storing null and empty values for the global secondary index key attributes. If the value of a global secondary index key attribute is null or empty, it is better to just skip the attribute when writing it

https://aws.amazon.com/ru/blogs/database/how-to-design-amazon-dynamodb-global-secondary-indexes/

So  all the methods that use UpdateItem api call (like update_fields, update or upsert) will lead to this error.

The only idea I have is to remove attribute instead of trying to set incorrect value (NULL/empty string) under the hood if this attribute is a key in GSO. We can use action DELETE instead of PUT https://docs.aws.amazon.com/en_us/amazondynamodb/latest/APIReference/API_AttributeValueUpdate.html#DDB-Type-AttributeValueUpdate-Action.

Does it make sense?

cc @pboling @richardhsu

[arekt]

No worries, thanks for quick response and your work on Dynamoid :) Deleting attributes without "present" values sounds good to me, but had no chance to dive in the library code yet. Something similar to what update_attributes do under the hood should work.