[FEAT] Output the vault secrets in a file

[sylvainmouquet]

Is your feature request related to a problem? Please describe. Currently we use GitLabCI and we export all keys of some paths in a json file. The arborescence in Vault is the same in the json file with a tree of keys.

Describe the solution you'd like The same feature with the GitHub actions input: a list of path output: a json with the key/values

Describe alternatives you've considered No one

[jasonodonnell]

Hi @sylvainmouquet, we'll take a look at adding this feature. In the mean time, contributions welcome!

[tcarrio]

@sylvainmouquet how do you expect this to be used? Will this be another way of calling the Vault CLI to output the JSON data?

An example use case may be useful for those who might be interested in implementing this feature.

I know it is possible to export secrets in JSON format using vault read path/to/secret -format json, for example.

[fean5959a]

Perhaps a solution may be to store the value of the key to a file. Usefull in some case if the value is a test file or a json. Perhaps having a syntaxe like: secrets: secret/data/<my_secret> my_key | file://<file_where_to_write_value_ok_key>

[fean5959a]

Suggestion (because I work with sylvainmouquet): I think it will be good (for this), to add 2 parameters:

• raw_output: (true|false)
• raw_output_file: </path/to/file>

"raw_output" keep initial json structure sent by Vault and actions provides it. The goal is to provide a json structure like this: { "<secret_1>":{ "key_1": "value_1", "key_2": "value_2" }, ... }

If we specify keys, only them will be provide. If we set "*" as key for a secret, all keys will be provide. if "raw_output_file" is set, json will be write to file, instead output action will be set.

[RichiCoder1]

In theory . should also work for "get all". So a theoretical implementation could be:

secrets: secret/data/key . > /tmp/secrets.json

With the > sigil signifying outputting to a file rather than an output variable.

[FalconerTC]

I'm interested in adding this, if there's still interest. Not sure which proposed solution would be better here though.

[romainvandecaveye]

i'm really interested by this.. Sorry, I meant I need this ! We are migrating from gitlab ci where you can just export a set of secret to a file, and we don't want to have to set all property manually

[slemme1]

I was going to open another enhancement request to read all secrets in the provided path without specifying each one in the secrets input: Example: secrets: /secrets/myPath * | prefix #get all secrets, using prefix+keyname as the output/env variable (or just keyname if no prefix). Adding this feature would address the issue of setting secret names manually. You can already gather all outputs and get them into json (and write to a file outside of the vault-action step): env: ALL_SECRETS_JSON: "${{ toJson(steps.vaultaction.outputs) }}"

I think this enhancement would just take changes in secrets.js getSecrets(): if {request.selector = *), remove this request and add a new one for each key in the path (populating request.envVarName, request.outputVarName as appropriate if prefix is supplied). I have not contributed to open source before and would be glad to take this on if I can get some mentoring on the process...

[mangatmodi]

Isn't it insecure? This enable people to cache/upload to artifacts. For now they only exist in variables in the runner process.

[minorgod]

Something like this would be very nice to have in situations where you work on a big team and ppl have not consistently named the vault paths so you end up with stuff scattered all over. I need to clean up a big vault such as this and it would be a huge help to be able to export everything under a given path (eg: /secrets/**/*) to a json file so I can see what the structure is, re-organize it, and then reimport it. Even if it only showed the path/key structure and obfuscated the actual secrets, it would still make my life easier. I'd love to see some sort of vault export/import that worked in json format. A "remap" or "migrate" feature might also be nice...something where you could dump the structure to json, then add some key/value such as "relocate-to": "/some/new/path" to a given path, then import it and it would move the paths to the new locations. Just brainstorming here. EDIT: Just found this project that seems to do a lot of what I'm looking for: https://github.com/jonasvinther/medusa

[iamludal]

Up, still needing this. :)

[gabyfulchic]

Up, still needing this! :+1: Having an argument to output secrets in a file with a post task to clean them after a workflow run would be crazy! If I can help too

[maxcoulombe]

Hey, chipping in on this with a different point of view :sweat_smile: . I agree having secrets in a file can be useful is many scenarios. However, I am not sure if it is the responsibility of the vault-action to convert the retrieved secrets into multiple different formats.

I generally prescribe to the philosophy of "do one thing and do it well" . I see vault-action has having 2 very vault-centric jobs and 1 single goal: authenticate to a Vault instance and read the secrets requested so they are available to the rest of the workflow. Whereas saving env variables into a json file seems like a vault-agnostic and generic job that'd best be done in its own step.

If the conversion was particularly hard to accomplish I'd see this differently, but as @slemme1 mentioned in his comment, it is not that much work to accomplish with the current vault-action implementation:

jobs:
  build:
    name: local-test
    runs-on: ubuntu-latest
    steps:
      - name: Vault Action
        id: vaultaction
        uses: hashicorp/[email protected]
        with:
          url: http://MY_VAULT_HOSTNAME:8200
          method: token
          token: MY_TOKEN
          secrets: |
            secret/data/test1 secret | VAULT_SECRET_1;
            secret/data/test2 secret | VAULT_SECRET_2;
            secret/data/test3 secret | VAULT_SECRET_3;
      - name: Save to file
        run: |
          touch secrets.json
          echo "${{ toJson(steps.vaultaction.outputs) }}" >> secrets.json
          cat secrets.json

In this case cat outputs:

{
  VAULT_SECRET_1: ***,
  VAULT_SECRET_2: ***,
  VAULT_SECRET_3: ***
}

The secret values are masked when printed but the file itself contains the real plain text values and can be used by subsequent commands or steps.

So given this responsibility does not quite fit with what the action is supposed to do, that supporting multiple formats in the action itself will inevitably lead to many follow-up requests for more formats and more options since there is never a one size fits all WRT output formats, and that this can be easily accomplished outside the vault-action itself, I'd lean towards closing this feature request.

Does his explanation and work-around make sense? Anything I am missing that supporting a json output format in the action itself would accomplish besides a bit of convenience because flipping a flag is easier than adding a step?

[RichiCoder1]

I would also encourage closing this, but with the caveat that this should be documented to also avoid any questions in the future.

[slemme1]

My original need was to export all in a generic way without knowing the keys. I uploaded the wilcard code in July 2021 but I don't think it made it to prod yet. Is there something else that needs to be done?

[maxcoulombe]

@RichiCoder1 agreed, let me know if you think this modification to the README would answer this question for future users: https://github.com/hashicorp/vault-action/pull/435

@slemme1 I'm looking at the PRs and issues throughout the repo and I know there are a lot of demand for more flexibility around wildcards and regexes when retrieving secrets. I'm trying to consolidate all requests and proposed solutions to either approve or submit a PR that'd take care of all demands. No promise on the timeline but it seems like the most asked feature of the whole project so it's important to me that we reach a conclusion, it's been pending for a long time.

[maxcoulombe]

I'll close this issue for now. If you think the workaround is inadequate and there are reasons to let the vault-action output the secrets into a JSON file directly feel free to re-open it and continue the discussion. Thanks for bringing this up.