sops icon indicating copy to clipboard operation
sops copied to clipboard

Piping using `/dev/stdin` doesn't work on Windows

Open MaikuMori opened this issue 5 years ago • 11 comments
trafficstars

The "workaround" to get piping to work is not cross-platform.

  • https://stackoverflow.com/questions/7395157/windows-equivalent-to-dev-stdin

I highly recommend handling it like kubectl and other tools where you can pass in - as file name which triggers reading from stdin.

So something like echo "asdf" | sops - or echo "asdf" | sops -f -

Is this something you would accept a PR for?

MaikuMori avatar Sep 14 '20 18:09 MaikuMori

@MaikuMori What is the use case for piping? Why not just write the output to a file and then run sops -e -i <filename>?

ajvb avatar Sep 14 '20 18:09 ajvb

I have a script which decodes content using sops -d enc_file and reads stdout. Then it changes the content dynamically and needs to update the original encoded file. Currently, I work around this by creating a temp file, writing the content there then running sops on it and then overwrite the original file, then delete the temp file. It would be much nicer if sops could just read stdin.

Even doing something like cat asdf | base64 | sops --output asdf.enc -e - in terminal would be nice. (Alternatively cat asdf | base64 | sops -e - > asdf.enc)

Here are other issues which mention this as preferred way:

  • https://github.com/mozilla/sops/issues/510 (mentions -)
  • https://github.com/mozilla/sops/issues/696 (could be solved with -)
  • https://github.com/mozilla/sops/issues/330 (suggested alternative solution, but you can clearly see that /dev/stdin is not expected behavior. People expect piping to just work or work with -)

To make this even more useful sops should use config file when input is passed via stdin (tracked by https://github.com/mozilla/sops/issues/510). To decide which config to use we could use CWD or the directory of output file if specified.

Maybe also defaulting input/output type to binary unless specifically specified.

MaikuMori avatar Sep 14 '20 19:09 MaikuMori

Additionally on windows, running in cygwin.

sops --decrypt --input-type yaml --output-type yaml --output /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml
#   Error: cannot operate on non-existent file

While

sops --decrypt --input-type yaml --output-type yaml - < /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml > /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec 

Would not.

jkroepke avatar Jun 05 '21 01:06 jkroepke

I'd like to add support for this feature. It prevents the necessity of unencrypted intermediate files that might potentially linger/be exposed on Windows. We are forced to work in a Windows environment, while the app is deployed to a Linux environment via CI/CD (also Linux based). We cannot use /dev/stdin and this request would provide a clean platform-independent way of reading via stdin.

iverberk avatar Aug 15 '22 11:08 iverberk

Just a quick comment for those who look a workaround, I used a temporary file with nanoseconds in the time to include the yaml content and remove it right after the sops execution.

if [[ $OSTYPE == 'darwin'* ]]; then curTime=$(gdate +%s.%N) else curTime=$(date +%s) fi

tempFile=./temp/${curTime}

ENCRYPTED_CONTENT=$(echo "$CONTENT" > $tempFile | sops -e --encrypted-regex "$ENCRYPTED_REGEX" --input-type yaml --output-type yaml $tempFile)

rm -f $tempFile

jodem avatar Sep 26 '23 15:09 jodem

If sops --encrypt by default outputs to stdout, why doesn't sops --decrypt read from stdin by default without a filename argument?

While the filename convention of using "-" to denote stdin is helpful, why not just read from stdin if no filename is given at all and stdin is not a tty?

erhhung avatar Feb 29 '24 00:02 erhhung