Bash process substitution not supported by the `--data-file` option

[omus]

Describe the problem

Bash process substitution isn't currently supported by this option:

❯ copier copy --data-file <(<<<'demo: foo') template demo
Error: Argument of data-file expected to be ExistingFile, not '/dev/fd/13':
    ValueError('/dev/fd/13 is not a file')

There seems to be an unnecessary file existence check.

Template

---
demo:
  type: str

To Reproduce

1. Run copier copy --data-file <(<<<'demo: foo') template demo

Logs

Error: Argument of data-file expected to be ExistingFile, not '/dev/fd/13':
    ValueError('/dev/fd/13 is not a file')

Expected behavior

Copier should accept temporary files this way.

Screenshots/screencasts/logs

No response

Operating system

macOS

Operating system distribution and version

macOS 15.4.1

Copier version

9.6.0

Python version

Python 3.10.13

Installation method

distro package

Additional context

No response

[sisp]

Makes sense to me. :+1: This doesn't work right now because the --data-file switch config uses Plumbum's ExistingFile type/validator which requires a path to be a file path but doesn't accept a character device. A possible upstream fix (or enhancement? 🤔) might be this:

-if not p.is_file():
+if not (p.is_file() or p.is_char_device()):

Or Plumbum could add a new type/validator for character devices, e.g.:

@Predicate
def ExistingCharacterDevice(val):
    """A switch-type validator that ensures that the given argument is an existing character device."""
    p = local.path(val)
    if not p.is_char_device():
        raise ValueError(_("{0} is not a character device").format(val))
    return p

Then, we could extend our --data-file switch config:

 @cli.switch(  # type: ignore[misc]
     ["--data-file"],
-    cli.ExistingFile,
+    cli.Set(cli.ExistingFile, cli.ExistingCharacterDevice),
     help="Load data from a YAML file",
 )

Would you like to raise an issue upstream in the Plumbum project, @omus?

[omus]

I can open an upstream issue but I suspect I'll only get to it tomorrow

[sisp]

It turns out that Bash process substitution isn't identified via Path(...).is_char_device() but via Path(...).is_fifo(). Thus, Plumbum needs the following type/validator:

@Predicate
def ExistingFIFO(val):
    """A switch-type validator that ensures that the given argument is an existing FIFO."""
    p = local.path(val)
    if not p.is_fifo():
        raise ValueError(_("{0} is not a FIFO").format(val))
    return p

But Plumbum's LocalPath has no is_fifo() method yet, so the workaround would be if not Path(p).is_fifo().

Now, knowing the terminology, I found https://github.com/tomerfiliba/plumbum/issues/337 which is requesting exactly this. I've tested this type/validator implementation with Copier, it works. However, it seems your CLI call was slightly wrong:

-❯ copier copy --data-file <(<<<'demo: foo') template demo
+❯ copier copy --data-file <(echo 'demo: foo') template demo

Using process substitution with a here-string doesn't work, but, e.g., <(echo '...') does.

[omus]

I don't think the here string is the problem as other CLI tools work with that syntax:

❯ cat <(<<<"demo: foo")
demo: foo

I still see the issue when I use echo:

❯ copier copy --data-file <(echo 'demo: foo') template demo
Error: Argument of data-file expected to be ExistingFile, not '/dev/fd/13':
    ValueError('/dev/fd/13 is not a file')

[sisp]

Perhaps that's a macOS vs. Linux (at least Ubuntu) difference. I'm on Ubuntu 22.04:

$ bash --version
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)

$ cat --version
cat (GNU coreutils) 8.32

$ cat <(<<<"demo: foo")

$ cat <(echo "demo: foo")
demo: foo

Yes, I wasn't intending to say that

copier copy --data-file <(echo 'demo: foo') template demo

is working now, just that the Bash process substitution might be wrong, but it seems it depends on the OS/Bash version.

[pawamoy]

Might be Bash vs. Zsh. The here-string example works for me in Zsh but not in Bash. And indeed MacOS users have Zsh by default IIRC.