Using bash Process Substitution silently fails...

[pmorch]

Describe the problem/challenge you have

TL;DR

ytt works fine if I have all my data files as real files in the file system:

generateSecrets > secrets.yaml
ytt -f values.yaml -f secrets.yaml -f demo.yaml
# Must remember this!
rm secrets.yaml

But if I use bash Process Substitution (short article), then the -f parameter is silently ignored.

This should be equivalent to the above (without using a pesky temporary file), but actually behaves as if the -f <(generateSecrets) wasn't there:

ytt -f values.yaml -f <(generateSecrets) -f demo.yaml

Please allow me to use Process Substitution. Or at the very least report an error so I don't think it works. But please make it work... The reason is that the secrets could come from a e.g. a vault or some other protected resource, and it would be great to avoid having to store them in temporary files.

Details

I have these files:

values.yaml:

#@data/values
---
secret: change-me

secrets.yaml:

#@data/values
---
secret: MySuperSecret

demo.yaml:

#@ load("@ytt:data", "data")

foo: bar
the-secret: #@ data.values.secret

This produces the expected output:

$ ytt -f values.yaml -f secrets.yaml -f demo.yaml
foo: bar
the-secret: MySuperSecret

But this does not:

$ ytt -f values.yaml -f <(cat secrets.yaml) -f demo.yaml ; echo $? 
foo: bar
the-secret: change-me
0

It silently behaves as if -f <(cat secrets.yaml) was not provided at all, but reports no error on STDOUT/STDERR or in $?.

[cppforlife]

because ytt is using file extensions to determine whether something is a yaml template or not, you have to "name" the file thats being ingested, ytt allows following syntax: ytt -f values.yaml -f foo.yml=<(generateSecrets) -f demo.yaml to name conent coming over the pipe.

[pmorch]

That was it. It works fine, actually. My mistake.

[pmorch]

But a little warning that the file was being ignored and why would've been helpful!

[damienleger]

I got there with a similar problem. I think the possibility of "naming" the file ingested should be added in the ytt --help.

[pivotaljohn]

@pmorch pointed out:

But a little warning that the file was being ignored and why would've been helpful!

This one is a tad tricky:

• the file is not completely ignored, but made available for use with data.read()... just like any file that does not have a relevant file extension (details in the description of the type file mark).
• given that the result of the process substitution is presented as a file, it's hard to imagine how to definitively distinguish these files from others.

Thoughts?

@damienleger suggested:

I got there with a similar problem. I think the possibility of "naming" the file ingested should be added in the ytt --help.

This technique is used frequently enough, assistance with getting it to work would be valuable to a good number of folks.