Add DynamoDB item Time to Live

[Dnomyar]

Hi folks,

I'd like to propose a change to add a TTL to each item to leverage DynamoDB's feature. I am not aware of any discussion about this subject.

To add more details, we've added some optional configuration for journal and snapshot tables to specify the TTL:

my-dynamodb-journal { // also available for snapshots tables
  dynamodb-item-ttl-config {
    field-name = "expiresAt" // in the AWS UI, we need to set the field name https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-how-to.html
    ttl = 30d
  }
}

With this configuration, it would add a field "expiresAt" for each journal item inserted with value now + 30d.

To facilitate the review process, this can be separated into two parts:

• the first part contains the bulk of the work: handling configuration & adding the field
• the second part contains the part that updates the alert related to the item size (<400k). Correct me if I am wrong, but I haven't found any similar alert for the snapshots.

I also tested it against dynamo in AWS:

Note: I am checking with my company for the CLA

Cheers

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[Dnomyar]

Thanks very much for your detailed feedback @coreyoconnor!

I pushed a few commits to try to address the feedback. I added a section in the README to describe how to use this and what are the things to be cautious about.

There is one thing I am not sure about in the readme. See my comment https://github.com/akka/akka-persistence-dynamodb/pull/99/files#r731892138

EDIT: Also, do you think Checking the expireAt during read. should be part of this PR or are you happy to raise an issue to implement that later on?

[coreyoconnor]

Thank you for your patience. I'm working to review this this week. Looks like I still have some CI fixes to push first.

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user

[lightbend-cla-validator]

At least one pull request committer is not linked to a user. See https://help.github.com/en/articles/why-are-my-commits-linked-to-the-wrong-user#commits-are-not-linked-to-any-user