spin icon indicating copy to clipboard operation
spin copied to clipboard

Implement AWS key value store

Open ogghead opened this issue 1 year ago • 0 comments

Hi folks! I am creating this draft PR to solicit feedback on an initial AWS key value store implementation. I appreciate any and all discussions on this PR!

Some points for thought:

  1. Implementation uses DynamoDB, though for large blob storage, S3 is preferable (DynamoDB can only store <=400KB size records), see https://github.com/fermyon/spin/issues/2606. DynamoDB is cheaper and faster for performing many rapid reads/writes of small amounts of data though, and is roughly in the same niche as Azure CosmosDB
  2. Auth currently requires generating AWS STS token credentials and passing them to the Spin app in a runtime config file. https://github.com/spinkube/skips/pull/9/files discusses better patterns to fetch credentials. Curious to hear thoughts on how this implementation can integrate better with that proposal!
  3. The Azure key-value implementation supports reading credentials from environment variables, however the AWS Rust SDK does not offer a synchronous API to load config and would require the MakeKeyValueStore::make_store function to be async for all implementations -- leading to a chain of async function coloring. It is possible to manually fill the SdkConfig object and I did this to pass STS tokens from a runtime config file, but it would be ideal to rely on the SDK's defaults and many credential loading fallbacks if possible. Curious on thoughts for how to best handle env var credential loading

ogghead avatar Oct 10 '24 14:10 ogghead