getsops
Breaking multiline formating in yaml as soon as a line end with a space
:wave:
As soon as a line in a multiline value end with a space, sops breaks the multiline formatting and convert to single line with \n
test: |
line one
line two with space at the end
line three
$ sops --version
sops 3.7.3 (latest)
$ sops -e test.yaml > test.enc.yaml
$ sops -d test.enc.yaml
test: "line one\nline two with space at the end \nline three\n"
The content of the YAML is identical, it's only the YAML formatting that changes. If you load both versions with a YAML parser, you obtain the same result. That's intentional, sops never promised to keep the formatting of YAML, only the content.
Sure, I know that it's the same content, but if I do:
test: |
line one
line two **without** space at the end
line three
$ sops -e test.yaml > test.enc.yaml
$ sops -d test.enc.yaml
test: |
line one
line two **without** space at the end
line three
As anyone would expected.
When using sops to store kubernetes secret configuration file encrypted in git for example, coming back to the configuration file and having it one lined definitely break the user experience.
definitely break the user experience.
I strongly disagree on that. The aim is to emit an equivalent YAML file containing the same information (including comments). That's what sops does, and what happens in your case as well.
If you want to retain formatting of the YAML file, you probably have to look for another tool. Alternatively let sops encrypt the YAML file as binary data, then you get the exact same content back.
So let me explain the full workflow with the real use case, maybe I'm doing it wrong:
I'm using sops with age to commit kubernetes secrets to git and let flux applying it.
I setup a .sops.yaml file in a parent folder with the public key and configuration to encrypt data part of secrets.
I'm using vscode-sops VScode plugin to call sops as soon as an encrypted file is open, to edit and re-encrypting file when closing it.
Using this workflow all I have to do to edit a secret in VScode is to open, edit, close, commit.
I hit the issue while trying to encrypt wireguard configuration file.
apiVersion: v1
stringData:
wg0.conf: |
[Interface]
Address = 10.43.0.1/24
ListenPort = 6530
PrivateKey = EM2K6hssd2G3snCZqk2LsAe/gzvhtwerMTbKCff7iVNvvv0tWI=
[Peer]
# srv42
PublicKey = faFB5CfMSzxlvfxvfdv7X7CZfFtudGVDr+YxnUitcyMuO57u6i9jU=
AllowedIPs = 10.43.0.2/32
kind: Secret
metadata:
name: wireguard-server
namespace: default
The workflow is working perfectly as expected, until I wanted to add a second peer so I open the file and prepare the configuration block. But I did not had the public key yet so closed the file as is:
apiVersion: v1
stringData:
wg0.conf: |
[Interface]
Address = 10.43.0.1/24
ListenPort = 6530
PrivateKey = EM2K6hssd2G3snCZqk2LsAe/gzvhtwerMTbKCff7iVNvvv0tWI=
[Peer]
# srv42
PublicKey = faFB5CfMSzxlvfxvfdv7X7CZfFtudGVDr+YxnUitcyMuO57u6i9jU=
AllowedIPs = 10.43.0.2/32
[Peer]
# srv43
PublicKey =
AllowedIPs = 10.43.0.3/32
kind: Secret
metadata:
name: wireguard-server
namespace: default
and re-opened it when I had the key but the unencrypted version is now:
apiVersion: v1
stringData:
wg0.conf: "[Interface]\nAddress = 10.43.0.1/24\nListenPort = 6530\nPrivateKey = EM2K6hssd2G3snCZqk2LsAe/gzvhtwerMTbKCff7iVNvvv0tWI=\n\n[Peer]\n# srv42\nPublicKey = faFB5CfMSzxlvfxvfdv7X7CZfFtudGVDr+YxnUitcyMuO57u6i9jU=\nAllowedIPs = 10.43.0.2/32\n\n[Peer]\n# srv43\nPublicKey = \nAllowedIPs = 10.43.0.3/32\n"
kind: Secret
metadata:
name: wireguard-server
namespace: default
Yes YAML content is technically identical, but it's now unmaintainable.
As you guest it, it would be the same for any plain secret file you want to push to kubernetes and any situation where you would leave a trailing space at any line in the configuration file.
@n0rad have you found a solution for this? This is unfortunate for maintaining yaml files in git.
Nope, I did look for a solution for now but it's ok for my own usage since I always double check each time by closing the file and opening it back to confirm the format.
On the other side, it's a kind of blocker for pushing sops at work for 250+ engineers where I will not ask them to do the check. I suppose we could implement some CI check to do validation but the workflow will be horrible. In the meantime we stay with sealedSecrets.
@n0rad you can store the Secret's values in sops-encrypted files and assemble them into a Kubernetes Secret by putting the following in your kustomization.yaml:
generators:
- |
apiVersion: viaduct.ai/v1
kind: ksops-exec
metadata:
name: secret-generator
secretFrom:
- metadata:
name: wireguard-server
files:
- wg0.ini
(This requires KSOPS.)
Since .ini is supported by sops, you can even encrypt only the private key by using the following .sops.yaml:
creation_rules:
- encrypted_regex: "^(PrivateKey)$"
Then your encrypted wg0.ini will look like this:
[Interface]
Address = 10.43.0.1/24
ListenPort = 6530
PrivateKey = ENC[AES256_GCM,data:erKILuevnjazTJoxzugfQ9Ss85t0nEaCbK7yyEyCPgJySc94bg+Hy9CQiZSt3f7gZHoX,iv:6gsdFlHDRjTdFJynQ8LEN98UkCDrjD9zDrKbggX4rcw=,tag:qNvwl6Axs6t3HOWR2verOQ==,type:str]
[Peer]
; srv42
PublicKey = faFB5CfMSzxlvfxvfdv7X7CZfFtudGVDr+YxnUitcyMuO57u6i9jU=
AllowedIPs = 10.43.0.2/32
[sops]
...
I just hit the same issue today on a completely different topic in helm resource rendering, making me thing the issue actually lives in go's yaml, more than sops
@n0rad The same issue occurs when using the helm values command. This happens when there is a space right before a line break in the yaml file
There's a space after "PublicKey=" in your yaml
[Peer]
# srv43
PublicKey =
AllowedIPs = 10.43.0.3/32
