YAML processing always strips quotes from literals

[silvpol]

I have reviewed other issues regarding preserving original file format and apreciate that retaining exact formatting of the source YAML is difficult. As a workaround, it would be very useful to be able to allow to force YAML output to always be quoted.

Our use case is Dropwizard YAML config file with env var substitution. Original file has quotes around passwords, which can be any special characters. After processing with SOPS they are gone and file ends up being invalid if password starts with any of the YAML control characters. Escaping injected values doesn't help since the quotes are already missing.

database: user: "${PGUSER:-justMe}" password: "${PGPASSWORD:-SuperSecret123}"

[felixfontein]

Our use case is Dropwizard YAML config file with env var substitution. Original file has quotes around passwords, which can be any special characters. After processing with SOPS they are gone and file ends up being invalid if password starts with any of the YAML control characters. Escaping injected values doesn't help since the quotes are already missing.

Can you provide an explicit example? If that's the case, it would be a bug in https://github.com/go-yaml/yaml/tree/v3, since we use that to serialize YAML files.

[silvpol]

I don't think this is a bug yaml.v3, more of a consequence of how it's processing and outputting. Our original file had quotes, but SOPS output doesn't retain them.

Original file

database:
    user: "${PGUSER}"
    password: "${PGPASSWORD}"

SOPS output

database:
    user: ${PGUSER}
    password: ${PGPASSWORD}

In Dropwizard we're using env var substitution. The problem happens when password starts with one of the YAML special characters like %, it ends up with invalid file even when encoded.

What we end up after substitution:

database:
    user: justme
    password: %123abcpass

If quotes could be forced or retained we would end up with valid YAML:

database:
    user: "justme"
    password: "%123abcpass"

[felixfontein]

Ok, so the problem aren't YAML control characters, but something Dropwizard does that's not conformant to the YAML spec?

[felixfontein]

sops produces semantically equivalent YAML files according to the YAML spec (assuming that yaml.v3 doesn't have bugs).

[silvpol]

It's more complicated than that, Dropwizard uses it as pre-parser so even if it endcodes characters it still doesn't help as encoding works only in quoted strings.

[mrwulf]

This can also change the value type: string

user:
   username: '123456'

becomes int

user:
   username: 123456

[felixfontein]

@mrwulf when I try out your example, the result is

user:
    username: "123456"

and not with an integer. What exactly did you do (and which sops version did you use) to get this behavior?

[yorik]

I have the same problem, I need to pass string:

key: "0x123456789abcdef"

and sops always strips quotes, so it become:

key: 0x123456789abcdef

and terragrunt fails to parse that with yamldecode:

Call to function "yamldecode" failed: on line 1, column 44: cannot parse "0x123456789abcdef" as tag:yaml.org,2002:int.

sops version is 3.7.3

[yorik]

Correction: it doesn't work only for long values:

key: "0x123456789abcdef1"

• works fine

key: "0x123456789abcdef12"

or longer gets stripped.

[felixfontein]

Here's the corresponding issue in go-yaml: https://github.com/go-yaml/yaml/issues/847