experimental_shell_command outputs do not preserve symlinks

[huonw]

Describe the bug Any symlinks in the outputs list of experimental_shell_command are resolved to normal files, duplicating the contents of the thing they link to. It'd be nifty to have the option to preserve them. At the very least, it'd be good to document this behaviour.

There's some cases where being a symlink (rather than a normal file) is be semantically relevant. For instance, with node, it seems the files within node_modules/.bin are symlinks into the rest of node_modules, and turning a link into a normal file causes those executable scripts to stop working.

Consider the following BUILD:

experimental_shell_command(
    name="make-link",
    command="""
    echo normal > normal.txt
    ln -s normal.txt link.txt
    # double check
    test -L link.txt
    """,
    tools=["ln"],
    outputs=["*.txt"]
)

experimental_run_shell_command(
    name="check-link",
    command="""
    cd {chroot}
    ls -l
    test -L link.txt || echo "not a link!!"
    """,
    dependencies=[":make-link"]
)

The key is make-link, while check-link is just a way to be able to introspect what make-link splats out when used as a dependency. The output of ./pants run //:check-link is:

total 16
-rw-r--r--  1 huon  staff  7 14 Sep 12:02 link.txt
-rw-r--r--  1 huon  staff  7 14 Sep 12:02 normal.txt
not a link!!

But I'd expect it to not have the not a link!! printed, and the ls line for link.txt should be lrwxr-xr-x ... link.txt -> normal.txt.

Full reproducer https://gist.github.com/huonw/4ebc739a7a04cdd9bcf137bdcbc2018f:

git clone [email protected]:4ebc739a7a04cdd9bcf137bdcbc2018f.git
cd 4ebc739a7a04cdd9bcf137bdcbc2018f
./pants run //:check-link

Pants version 2.13.0 2.14.0rc0

OS macOS

Additional info Thanks for pants! 👖

[stuhood]

Thanks for the great description!

@thejcannon is actually actively working on this one via #10271: see #16844.