Add `makeself` backend

[grihabor]

Makeself is a small shell script that generates a self-extractable compressed tar archive from a directory. This backend provides a makeself_archive target which can package a makeself archive. Here is an example:

src/shell/run.sh:

#!/bin/bash
src/shell/hello.sh
src/shell/world.sh

src/shell/hello.sh:

#!/bin/bash
printf hello

src/shell/world.sh:

#!/bin/bash
printf world

src/shell/BUILD:

shell_sources()

BUILD:

makeself_archive(
    name="hello",
    startup_script="src/shell/run.sh",
    files=["src/shell/hello.sh", "src/shell/world.sh"],
)

Then you can run pants package :hello and it will produce an executable dist/hello.run (the file extension is recommended by makeself maintainer). Here is an example repo.

This is a minimal implementation and does not yet support all makeself options.

[huonw]

FYI @grihabor, I'm going to be on leave for 2 weeks from tomorrow. Please feel free to work through my remaining comments and/or solicit more reviews on #development on slack. Thanks for taking the time to contribute!

[grihabor]

Ok, thank you!

[grihabor]

Rebase changes: removed docs related stuff because it was moved

- build-support/bin/generate_docs.py                 |   3 +-
- .../Using Pants/concepts/enabling-backends.md      |   1 +

[grihabor]

@huonw @kaos What do you think? Shall we merge this?

[huonw]

Thanks for waiting. I think this is fine to land as experimental (as it is) pending @kaos's change. I'd like to see some more thinking on https://github.com/pantsbuild/pants/pull/20360#discussion_r1439897543, because I don't think requiring all binaries to exist is a viable path forward for any sort of usage, beyond initial experimentation.

(There's a chance the new tests here will break someone running pants test :: if they don't have the right local setup... but running that should be fairly rare, I think.)

[grihabor]

@kaos @huonw Thank you for your review!

It turned out that makeself supports not just a single shell script, but arbitrary command with arguments. So I changed the API and I no longer need os.curdir.

The changes (https://github.com/pantsbuild/pants/pull/20360/commits/687e1685d3da22ed31485fd64edf04c862cb3353):

• added makeself_archive.args for arbitrary makeself args
• makeself_archive.startup_script is now a list of tuples, for example, ["./my_script.sh"] or ["echo", "pants"]
• added more tests

[grihabor]

P.S. ready for the next review round.

[grihabor]

@huonw Shall we merge this one?

[huonw]

Thanks for waiting.

I tried running these tests locally, without zstd available, and observed that tests fail:

pants.core.util_rules.system_binaries.BinaryNotFoundError: Cannot find `zstd` on `['/bin', '/opt/homebrew/bin', '/usr/bin', '/usr/local/bin']`. Please ensure that it is installed so that Pants can zstd file.

So, I think we probably can't land this directly, or else we may be breaking running Pants tests on some platforms. Some options are:

1. ensure that the long list of system binaries are optional before landing
2. require that zstd, gpg, ... are all installed to work with this part of the Pants codebase (i.e. document it and communicate on slack)
3. something else?

In addition, once we've got a direction here, you'll need to merge main and add some release notes to docs/notes/2.22.x.md. See https://github.com/pantsbuild/pants/discussions/20888 for more info.

[grihabor]

@huonw Thanks for testing! I've reduced the list of binaries to only required ones and added the tools field to add more binaries if needed. Also added notes and a documentation page near the shell docs.

[grihabor]

Hey @huonw, could you take another look? Thanks

[huonw]

(Just being explicit: approval conditional on the copy-editing and answering that question, in case that wasn't clear. 😄 )