Way to "dry-run" a decrypt

[james-callahan]

trafficstars

For file validation purposes, we'd love a way to "dry-run" a decrypt, so that given e.g. a sops-encrypted YAML file; everything that would be decrypted, would instead be replaced with *********

Basic example

example.enc.yaml:

example_key: ENC[AES256_GCM,data:GuFvI4uWtnutMgBE1w==,iv:rYPH8/XFF1oBEeo+hzHkjr8yyn1SDObITONbQLqrMhE=,tag:cuvAbQQBAYKCEcMlJk3W6w==,type:str]
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age: []
    lastmodified: "2022-03-03T03:39:40Z"
    mac: ENC[AES256_GCM,data:1v6SMIebvS7VB9XFzWWP4Ti1r2JQea/lGGSRTKSfOUaF9TXH7eRRxx9ociH/F/iq72DOZV940tZZXxb42dSkLBssVRWOA+vcEOhGK9KdUIci7pgt2wSL/n3cxUc+f0Vr1bfHW/aMsW7oJyYDNQRIzEKMXUVem+Sxai7+w0gw8nE=,iv:ET0n680qIH8TJ+8Lt/t0Atm9pro9VsKa+eUJTqEMsgQ=,tag:aN6w/BjuZpgnzSTpX/2OUA==,type:str]
    pgp:
        - created_at: "2022-03-03T03:39:27Z"
          enc: |
            -----BEGIN PGP MESSAGE-----

            hQIMA3lxnYU9D1ZiAQ//dWNV3iTDe5YSJRz7+mzMBWt7kJ4PWWizNvV+iEtCnHaq
            TZdfFmS6LPwkXDKu7Z8B/b8ev+TIxdM0JBbpfPRxj09N/Sh9pmYjl1tLQpuKx9KZ
            vtEPDbzTlIXyuhBBBbWLZYsB0ImOGIIbrxIdeB80+kGGQU94isBdVWEqHoEzPDk5
            /jJF6AxgOMCf5sZ+/sYNNMl3Mf0l+NwI6TfV7Pqd6lbQuxs4umIsYMCj3O3eatYh
            6g5wDPvnZNSE0m3DQKPMaI21fRkJRyexbjkYFx6WYUwbAjZ8Tpxi7orkOdiPtiMn
            IAdxIIT0T7kVGzk4IMB5tTLXcs09FbBWo0x6q5raHeVuYG19s0bjGYx0wfi4N+gH
            bCsqnYj/klWxR/+KJ2fIqX6kx1vEphhMdwCXFP/nkX9OCF/l9x8EkWo4gUhpUPT5
            mFbfLJulQNKzQxMq8Zds75HaUtMxT1WS2nboJS05iDfd/M+JdUMsz8Nc9NPbTV82
            tZjyp2KwvcG7jij4UcqUxfu+tb0h4qM8o9m4WbWFap5gPC583btyrZZfdebbDjVp
            cVoCT8yTz349NxueQXiR/EYYN4TbT84Se6SiogzlFyk/1eEA2YFcm66/dgmsvDZz
            6Ka7PSgsWL4U+ifJBHJGzToP7NOm4ESRj24WslBeK6PQumYNy1AM1Kv9eUFiBc/S
            XgHYRlewZRrLz5Lsl1RuHL9VBRZ716KGWNQDHGlQGccQBMUTvp2AlbvnSlGByNvN
            U5vB1Cy3g3bT5knBSQOjVwylbstyHAe618fttY6sJVLLQ00V8EH9k14247/4m0A=
            =nmOn
            -----END PGP MESSAGE-----
          fp: C67898BEDF14E103FDFDB067A37403A776EBF4E5
    unencrypted_suffix: _unencrypted
    version: 3.7.1

sops -d --dry-run="******" example.enc.yaml output:

example_key: "******"

When an `encrypted_regex` or other field is present, that would be respected.

foo: ENC[AES256_GCM,data:rZN1,iv:57K06DrR1eOGFUTlux/91WiPc95c3enruqRhm6uh5ok=,tag:xZNyP+8tyRaOYGEsWKaufQ==,type:str]
bar: bar
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age: []
    lastmodified: "2022-03-03T03:41:16Z"
    mac: ENC[AES256_GCM,data:mm1hLjZl23TgXHhUv5hwRYxR6cO3pRamS0ktggoJ5k3z2wVbxh24tNcU2X4cChz8K7NFq4mAwZR77GOpoYv5pHt4gsjc09T2NUQkFPKSxKrxjGhpRcBVKXArPS8V61mAY7qOZsVcasEmFgemO+6uNVSh9wk1vKy0VIMbZ9RjijE=,iv:jYH+D/lLnG3J02mjr/fTFIPyW9DCaMNW9CiZG4NTI7Y=,tag:R63f1hXAV3bTSXSLRZ5/oQ==,type:str]
    pgp:
        - created_at: "2022-03-03T03:41:03Z"
          enc: |
            -----BEGIN PGP MESSAGE-----

            hQIMA3lxnYU9D1ZiAQ/+OY4J8cBgDEmlfzt37TOqFcz3p5o5ZrQupPTCkAJn90ZM
            S+ZXcfBJK3ZnxokB7dSV5aY4UVpoV3a83SEUeO5p3zShlMR/SvOwpYa+IdHaaL0A
            A/POGnYK0aI8DMl7J0KX2MofiLHuQlSxmDatQBH6wvTO078VdkIrhBJZLdNweCiV
            xFus3IvtAsgx9lQ3wzm7eXYcGhXwhvGBoEEm7kI9B8l9lrF+WKsqUTmyoU6SZh9Y
            4qXxwdkAclFx5bSbVHpI4t6L5/e4tzB7lRlxXH4hh/sXnpZRWc/c0tUYoS/Hecss
            3vGcmrQNmhnzpFuFztJV8pim6TLnSi6IPX3puVYDfoOCWyV9cxUpD0Ry/658cmFv
            ND0urR0LgCRQovI6l7BVDywruGq58HH1tPWwN7PtUsiJpSDysaMqp1vZj3BhENgI
            sMr6+dJhL+mY25EQrKGyXO0W36Yb3Jm2CjI+TjawwswQ0nzYpR4RqtBXwh7lRBaw
            MWQvyNLRdN7dWjxfo+RkP8w1bhIzLR7G2imNRa8IkurEqvtR6iqfgMDlKR8JQVES
            fwLgtuz94WGx86XICWjg3CVzJAavLStK8AdRD815iwYC9Wuui1ylDQ7IB67vjDKO
            udFYb8o97hdXrot7ewwZH1g64G/RJivVpU5AYU0C0cjohZmFnvnKtFEQGJ8PAYPS
            XAEbb6vVQw0X7xQSuww7INQaRCyF6LLCnro2C9+zsbFoc4iisomu9QL+qYGJP01N
            VnFNHGsL405lvQrDkHgFnx2fOXHdmZWZtmL/NG+QaB3+6FpneHaHmsiQyjiT
            =Bu76
            -----END PGP MESSAGE-----
          fp: C67898BEDF14E103FDFDB067A37403A776EBF4E5
    encrypted_regex: foo
    version: 3.7.1

sops -d --dry-run="******" output:

foo: "******"
bar: bar

[ajvb]

@james-callahan can you describe your use-case for this?

[james-callahan]

@james-callahan can you describe your use-case for this?

I'd like people to be able to validate their sops encrypted files; without granting access to the actual values.

One destination I'd like to get to, is running ksops in this newly proposed dry-run mode; and then validate the kubernetes manifests that come out.

[ajvb]

I'm not a fan of this. First off, if someone has the ability to decrypt a value then they have access to it. If this is purely about decrypting a value remotely then redacting the display of it, that should be implemented within the system displaying the value.

[james-callahan]

if someone has the ability to decrypt a value then they have access to it. If this is purely about decrypting a value remotely then redacting the display of it, that should be implemented within the system displaying the value.

No, the intention is a way to get a "sample" (in terms of fields present) of a decrypt when you don't have the key/ability to perform a decryption.

[james-callahan]

Have you considered this further at all?

[br-arruda]

Hi, a bit too late on this topic.

Recently I had to write a pipeline with SOPS, and I thought a dry-run could help me to check what files were encrypted. For context, the folder with YAML had both encrypted files and without encryption. At first I thought something like:

for file in path/to/manifests/*; do
    sops -d --dry-run $file && sops -d -i $file
done

but I just realized that as I would test all the files, actually I could just rewrite this as the operation itself:

for file in path/to/manifests/*; do
    sops -d -i $file || true
done

Usually I use dry-run in other tools to validate that an operation could have success, but I think SOPS doesn't have to work this way, because it is a All-or-Nothing operation, there is no possible way to validate a symmetric cipher without really doing the decryption itself.

[james-callahan]

I think SOPS doesn't have to work this way, because it is a All-or-Nothing operation, there is no possible way to validate a symmetric cipher without really doing the decryption itself.

A dry-run of sops wouldn't do the decryption; but instead

• validate that the file is syntactically correct (e.g. not invalid yaml/env file)
• validate the contents of the sops field matches the sops schema (e.g. has a mac sub-field)
• remove the sops field that sops adds on output and fix other escaping/renames it does

[felixfontein]

@james-callahan I don't think what you are describing should be called dry run. That's a kind of lint operation for the sops file format, not a dry run for decryption (which would be a decryption without printing the result).

I'm also not sure what your last bullet point should actually do, this sounds like a reformatting tool, and not like a lint or dry run operation. Or are you simply describing a YAML/JSON/env/... parser?

[br-arruda]

Something like a sops --validate <file> (not a dry-run) would be useful for use cases like this one about pre-commit hook validation: https://github.com/onedr0p/sops-pre-commit/issues/24.

As SOPS doesn't have such functionality, the project above relies on custom logic that is restricted to the underlying structure of K8s Secrets. It would be useful for script integration if SOPS could offer a command to signal that a file looks like an encrypted file, or that a file doesn't have the field described in the suffix/regex command flags.

[james-callahan]

I'm also not sure what your last bullet point should actually do, this sounds like a reformatting tool, and not like a lint or dry run operation. Or are you simply describing a YAML/JSON/env/... parser?

When sops encrypts a file, it adds a sops element to it to keep the metadata for itself.

When sops decrypts a file, it doesn't show the sops element in the output (implemented at e.g. https://github.com/getsops/sops/blob/c6ce6dfcae44032f6a9b5e9166f44764534f301d/stores/yaml/store.go#L285). When running a dry-run, sops should equally not show the sops element in the output.