--from-env-file Implementation to kubernetes_secret resource

[albertorm95]

Description

I want to be able to use kubectl functionality when creating a secret with flag --from-env-file what does this do is to iterate over the input file and line by line (Code) split the = from the line (Code) so you can have the key-value pair and use those values as the data map[string][]byte.

Potential Terraform Configuration

Code in resource_kubernetes_secret.go should be modify so now it can accept an env-file field.

Schema: map[string]*schema.Schema{
    "metadata": namespacedMetadataSchema("secret", true),
    "data": {
        Type:        schema.TypeMap,
        Description: "A map of the secret data.",
        Optional:    true,
        Sensitive:   true,
    },
    "type": {
        Type:        schema.TypeString,
        Description: "Type of secret",
        Default:     "Opaque",
        Optional:    true,
        ForceNew:    true,
    },
    "from_env_file": {
        Type:        schema.TypeMap,
        Description: "Path to a file to read lines of key=val pairs to create a secret.",
        Optional:    true,
        Sensitive:   true,
    }
}

So then we can use:

resource "kubernetes_secret" "foo-env" {
  metadata {
    name = "foo-env"
  }
  from_env_file = {
    "foo-env" = "${file("foo.env")}"
  }
}

References

I did study how kubectl does this solution and I might be able to apply that in Terraform, the process in kubectl is:

Input foo.env:

TEST=teststring
NODE_ENV=development

kubectl create secret generic foo-env --from-env-file=foo.env

Then -> create_secret.go Complete() -> secret.go StructuredGenerate() -> secret.go handleFromEnvFileSource() -> env_file.go addFromEnvFile() -> env_file.go proccessEnvFileLine() -> secret.go addKeyFromLiteralToSecret()

Output Kubernetes secret object created with flag --from-env-file:

apiVersion: v1
data:
  TEST: base64string
  NODE_ENV: base64string
kind: Secret
metadata:
  name: foo-env
  namespace: default
type: Opaque

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jrhouston]

Thanks for opening this @albertorm95 this is an interesting issue, and I think there are a few paths forward here:

1. It's possible to do this purely with Terraform's built-in functions, as the data field of secret is of type map:

resource "kubernetes_secret" "test" {
  metadata {
    name = "test"
  }

  data = {
    for line in compact(split("\n", file("vars.env"))):
      split("=",line)[0] => split("=",line)[1]
  }
}

This will work for now, but it's pretty ugly and will lead to copypasta if you use this a lot. This approach also has a few gotaches, i.e if the values in your dotenv contain = or spread across multiple lines.

2. Like you suggested, we could add support for this in the provider by adding a from_env_file attribute that will parse the env file format for us. We're open to accepting a PR that implements this feature so we can do something like this:

from_env_file = file("vars.env")

We will have to add this to both ConfigMap and Secret.

3. Adding envdecode() to Terraform's set of functions for encoding/decoding files. This would mean we could do something like this:

data = envdecode(file("path.env"))

This feels like the most reusable way to enable this sort of behaviour, but for this we would have to open a PR against Terraform's core repo.

[albertorm95]

Ok, so let me then create an Issue in the core repo in order te add the envdecode() function to Terraform's set of functions for encoding/decoding files.

So once thats done I could use it to resolve this issue!

[jrhouston]

So once thats done I could use it to resolve this issue!

It takes a fair bit of time to get something into Terraform core (if they are willing to accept it) so I am happy to help add this to this provider in the mean time.

[jrhouston]

Looks like there is already an open issue about this on the main Terraform repo, see here: https://github.com/hashicorp/terraform/issues/23906

[cameronspeakflash]

Any progress toward this? I've got the for loop in my terraform files but would be nice to simplify that!

[hadim]

I would be interested in that feature as well.

[jrhouston]

I have an open PR for this on the core repo, add your 👍 there if you want to see this feature.

[jrhouston]

It occurred to me that we could solve this use case by creating a utility provider with a data source that can read the .env format. As there is some contention about adding a dotenvdecode function to terraform core I've created a dotenv utility provider here: https://registry.terraform.io/providers/jrhouston/dotenv/latest/docs that can be used to feed .env files into configmaps and secrets.

Here is an example:

terraform {
  required_providers {
    dotenv = {
      source  = "jrhouston/dotenv"
      version = "~> 1.0"
    }
  }
}

provider kubernetes {
  config_path = "~/.kube/config"
}

data dotenv dev_config {
  # NOTE there must be a file called `dev.env` in the same directory as the .tf config
  filename = "dev.env"
}

resource kubernetes_config_map cm {
  metadata {
    name = "example"
  }

  data = data.dotenv.dev_config.env
}

[github-actions[bot]]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[github-actions[bot]]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[jrhouston]

I'm reopening this one as we could now add this to the provider as a Provider Defined Function.

We could add a envdecode function that replicates the functionality of the --from-env-file logic here: https://github.com/kubernetes/kubectl/blob/8d3b6cf1408476c8421eb5d4bf26b1d7badec47c/pkg/generate/versioned/env_file.go#L35-L70