temporal icon indicating copy to clipboard operation
temporal copied to clipboard

Temporal assumes the credentials of storage service used for archival will stay the same through out the life cycle of the application.

Open VikasNS opened this issue 3 years ago • 2 comments

Expected Behavior

  1. We use AWS S3 to store archived workflows.
  2. Authentication is via AWS Keys. The keys are pulled every 8 hours and written to the credential file. This is done as our keys are valid only for 8 hours.
  3. I expect temporal to use the latest keys from the credential file each and every time when connecting to AWS S3.

Actual Behavior

  1. Archiver object which does the archiving (which containers the AWS session object) is created only once for the first call and is cached. There after for subsequent calls, the same archiver object is used.
  2. This means that the AWS Keys present at the first call (at the start of the application) is used for the whole life cycle.
  3. This will result in archival failure after 8 hours and the keys expire after 8 hours.

I feel the root of the problem is that temporal assumes that the credentials will stay the same for the whole lifecycle of the application which isn't true.

Steps to Reproduce the Problem

  1. Setup AWS S3 archival
  2. Configure the AWS Keys to expire , say 4 hours from creation.
  3. Archival will stop working after 4 hours.

This has been discussed here -> https://community.temporal.io/t/aws-keys-updated-every-10-hours-in-credential-file/3612

Specifications

  • Version: 1.1.12
  • Platform: Java

VikasNS avatar Feb 05 '22 12:02 VikasNS

This is experimental feature, we will address this once we have resource to work on productionize this.

yiminc avatar Feb 11 '22 22:02 yiminc

Not sure that this use case is supported by the aws go sdk yet. See this discussion for more context and a possible workaround: https://github.com/aws/aws-sdk-go/issues/1993

jontro avatar Jun 28 '22 16:06 jontro