terragrunt-atlantis-config icon indicating copy to clipboard operation
terragrunt-atlantis-config copied to clipboard

Avoid calling sops_decrypt_file function

Open povils opened this issue 2 years ago • 3 comments

Currently terragrunt-atlantis-config is actually invoking the terragrunt function sops_decrypt_file. When we have thousands of small secret files with sops_decrypt_file, it would be great that the terragrunt-atlantis-config would just skip the invocation. I don't really see a need to be honest, because not sure how reading secrets is helping to autogenerate atlantis.yaml. Besides unneccesary calls to let's say AWS KMS in order to decrypt files, terragrunt-atlantis-config command also requires AWS access in the first place

povils avatar Sep 07 '21 15:09 povils

There are many terragrunt functions that are necessary to generate atlantis.yaml, but I agree that sops_decrypt_file is not a super common one that would be needed. It's possible we could add a block list for functions that we don't want Terragrunt to actually execute (like I have started in a WIP PR: https://github.com/transcend-io/terragrunt-atlantis-config/pull/70/files#diff-6a86d8a9f6cd199e4c88a751a77dc763a1ad3a8d31aa7f2b3378a82f7f2cc4ffR136-R158).

Are the permissions here the biggest issue for you, or is there performance related data you have on this taking too long for your tastes, or are you being rate-limited by KMS?

For the permissions, where do you have terragrunt-atlantis-config running? In most cases, I would have guessed that the tool would be run on the Atlantis server, which would need those permissions anyways in order to actually run the atlantis plan commands right after generating the atlantis.yaml file. I'd gladly hear your use case though!

dmattia avatar Sep 07 '21 19:09 dmattia

Hi, thanks for getting back! Permissions I would say is not the biggest issue, as you said atlantis still needs it. It's more of a question if you change only a staging file which affect a specfiic staging account, why we would need to decrypt all secrets unrelated to other dozens accounts.

Performance is an issue, it's managable but as I said maybe it's a low hanging fruit if we indeed don't need the depcrytion invocattion

povils avatar Sep 08 '21 07:09 povils

Added a help wanted label. As this isn't a correctness thing nor a permissions related issue in my mind, I may not have time to work on it, but would gladly review a PR that does so

dmattia avatar Feb 25 '22 20:02 dmattia