terratest
terratest copied to clipboard
gruntwork-io
Sensitive outputs shouldn't be exposed when using terraform.OutputRequired
Hi,
We are working on a test-case that creates some IAM users and access keys that we use for authenticating with some AWS resources within that test. To be able to use the credentials we pull them from the outputs, e.G:
outputs.tf:
output "aws_iam_access_key_secret" {
description = "The acccess key secret."
value = aws_iam_access_key.docker.secret
sensitive = true
}
test.go:
accessKeySecret := terraform.OutputRequired(t, terraformOptions, 'aws_iam_access_key_secret')
Using the helper methods to retrieve outputs with terraform will dump the outputs to the logs. I believe we should align the behavior of terratest with terraform and only write <sensitive> instead of the actual values to the logs.
Ok, so running terraform output will mask any outputs that have sensitive = true set. If one calls terraform output on a specific output ( e.g. terraform output aws_iam_access_key_secret terraform returns the value of the output as clear text, which makes kind of sense to me also.
It seems to me that there is no easy way to be aware if we deal with sensitive data or not when calling terraform output on a specific output. The log line is written in the command.go file though. It might be a reasonable and simpler approach to make the logging of the output configurable.
wdyt @brikis98 ?
Try terraform output -json. I believe that shows you which ones are sensitive.
yeah sure, terraform output also shows you which ones are sensitive. That isn't very helpful though since the outputs methods in terratest would end up calling terraform output key, which will always return clear text.
Yea, what I meant was that the terraform.Output(xxx) helper would always run terraform output -json with no logging, read output xxx from the JSON, and only log what it saw if it wasn't sensitive. So it's making the logging optional in command.go as you said, but also making the terraform.Output helper in how it uses it.
Thank you, that makes sense to me! I will take a proper look in a bit.
We've been seeing this issue as well. We apply some terraform, and then use terratest's output functions to fetch out a randomly generated password. We need the password to ensure that the applied resources have been set up, but by getting the output, it also prints out the password to our logs - not ideal.
It feels like by default logging should be turned off for any sensitive output.
Has there been any progress on this issue?
Here is a workaround for this that we recently identified:
func fetchSensitiveOutput(t *testing.T, options *terraform.Options, name string) string {
defer func() {
options.Logger = nil
}()
options.Logger = logger.Discard // import path is github.com/gruntwork-io/terratest/modules/logger
return terraform.Output(t, options, name)
}
I started using terratest to test my Kubernetes Webhook (k8s-image-swapper). The IAM keys and docker credentials were exposed in the logs, which is not ideal for CI purposes. I came up with the following solution:
Usage:
func TestHelmDeployment(t *testing.T) {
workingDir, _ := filepath.Abs("..")
awsAccountID := os.Getenv("AWS_ACCOUNT_ID")
awsRegion := os.Getenv("AWS_DEFAULT_REGION")
awsAccessKeyID := os.Getenv("AWS_ACCESS_KEY_ID")
awsSecretAccessKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
ecrRegistry := awsAccountID + ".dkr.ecr." + awsRegion + ".amazonaws.com"
ecrRepository := "docker.io/library/nginx"
logger.Default = logger.New(newSensitiveLogger(
logger.Default,
[]*regexp.Regexp{
regexp.MustCompile(awsAccountID),
regexp.MustCompile(awsAccessKeyID),
regexp.MustCompile(awsSecretAccessKey),
regexp.MustCompile(`(--docker-password=)\S+`),
},
))
...
Implementation::
type sensitiveLogger struct{
logger logger.TestLogger
patterns []*regexp.Regexp
}
func newSensitiveLogger(logger *logger.Logger, patterns []*regexp.Regexp) *sensitiveLogger {
return &sensitiveLogger{
logger: logger,
patterns: patterns,
}
}
func (l *sensitiveLogger) Logf(t terratesttesting.TestingT, format string, args ...interface{}) {
var redactedArgs []interface{}
obfuscateWith := "$1*******"
redactedArgs = args
for _, pattern := range l.patterns {
for i, arg := range redactedArgs {
switch arg := arg.(type) {
case string:
redactedArgs[i] = pattern.ReplaceAllString(arg, obfuscateWith)
case []string:
var result []string
for _, s := range arg {
result = append(result, pattern.ReplaceAllString(s, obfuscateWith))
}
redactedArgs[i] = result
default:
panic("type needs implementation")
}
}
}
l.logger.Logf(t, format, redactedArgs...)
}
@brikis98 What are your thoughts on this? Do you think this is something we can add to terratest itself?
