argo-events icon indicating copy to clipboard operation
argo-events copied to clipboard

feat: AWS Temporary credential support for SQS eventsource

Open harshdeep-23 opened this issue 1 year ago • 2 comments

Checklist:

  • [ ] My organization is added to USERS.md.

fixes #1901

harshdeep-23 avatar Jul 21 '22 13:07 harshdeep-23

I'm trying to understand this PR:

  1. It added a new filed for sessionToken, referencing a secret, which might be expired very soon;
  2. If the sessionToken expired, it automatically send a request to generate a new token, and use it for connection, while the new token is not persisted anywhere;
  3. Every time the eventsource pod restarts, it repeats the same process.

Am I correct?

whynowy avatar Jul 25 '22 23:07 whynowy

@whynowy Generating new credentials is not handled by the eventsource. Rather it is assumed that if the temporary credentials have expired, there is some external process which keeps updating the k8s secret with a valid credential. When the eventsource encounters credentials expired error from AWS, it tries to read the mounted credentials again (which might have changed by the external process). The issue has more details - https://github.com/argoproj/argo-events/issues/1901

harshdeep-23 avatar Jul 26 '22 19:07 harshdeep-23

What do you think if we use another approach, that we provide an option to configure how ofter it reloads the secrets?

@harshdeep-23 - what do you think?

whynowy avatar Aug 07 '22 07:08 whynowy

What do you think if we use another approach, that we provide an option to configure how ofter it reloads the secrets?

@harshdeep-23 - what do you think?

@whynowy I'm assuming you mean like a configurable way to periodic reload the secrets. If we go with that, then it might happen that the credentials expire and the requests to pull from SQS keep failing until the next secret reload cycle. We would have to keep the reload frequency very small to quickly recover if the credentials expire. I think it would be an overhead to keep a thread/coroutine continuously reloading secrets (like every X seconds) vs reloading every few mins/hours when the credentials actually expire.

harshdeep-23 avatar Aug 07 '22 09:08 harshdeep-23