Add support to extract archived files entirely

[marcosnils]

Updated 02-02-22

When bin finds an archive type as a source, it currently lists the files inside and prompts the user which file to extract.

There are several projects like https://github.com/kubernetes-sigs/kui which requires extracting the complete archive and then add a specific binary to the PATH variable. It'd be great if bin could help with this workflow. i.e

$ bin install <some_package.zip>

some_package has been installed in $PATH/.some_package (hidden folder maybe?), executable selection: 
[1] some_package/packagebin
[2] some_package/packagebin2
[*] All executables
Select option: 1   # This could be 1,2 (list) 1-n (range) or * for all?

# After this selection, `bin` would make a symbolic link of the selected executables in the `default` config path.

some_package has been installed successfully.

What things should bin do to provide this experience?

• Check if the root file of the archive file is a folder so it creates a hidden(?) directory in case it's not the case to avoid extracting files spread all over the bin default path.
• After extracting, bin should scan all files and folders that contain binaries and display them for the user to select
• bin should create symbolic links of the selected files in the default_path config dir.
• We need to implement symlink creation in all platforms

note: This feature is breaking UX compatibility since the default way of installing archive files would be through the experience highlighted here instead the current one (selecting one single file). I'm personally ok with that since I believe this new experience is way better for this use case.

[stealthybox]

Alternatively, could we just have bin symlink each of the binaries into ~/bin? This would be similar to brew link

containerd/nerdctl is another great example of a multi-binary package with other important assets:

bin install github.com/containerd/nerdctl
   • Getting latest release for containerd/nerdctl

Multiple matches found, please select one:

 [1] nerdctl-0.16.1-linux-arm64.tar.gz
 [2] nerdctl-full-0.16.1-linux-arm64.tar.gz
 Select an option:

[marcosnils]

Alternatively, could we just have bin symlink each of the binaries into ~/bin?

I like this approach as well. What I like about this is that we don't need mess with the user's PATH or instruct them to do any further step after extracting.

I'll update my 1st post with the new proposal

[rothgar]

Funny enough, I created a brew formula for nerdctl because I didn't like having to do the extraction/linking every time https://github.com/rothgar/homebrew-tap/blob/master/Formula/nerdctl.rb

I think symlinking all the binaries sounds fine (so long as a file doesn't already exist). Only thing I'm not sure about is if we need the extracted folders to be hidden. I'd be fine with a regular folder since that's where the symlink will be pointed anyway.

[skorokithakis]

I realize this is a niche use case, but I use Syncthing to synchronize my bin directory, and this flow would kind of break some stuff. I can switch away from Syncthing for bin-handled files, since I already sync the config file and it has bin ensure, so that's not a big issue. However, I think it would be cleaner if it used a temporary directory (ie with mktemp) to store the files before copying the final output to the path.

This avoids storing the entire contents of the archive when all I want is a single binary. However, I do recognize that this more general method gets basically every case and works well both with programs that require the extra files and those that don't.

I just want to request that the symlinks are relative, and not absolute, so syncing programs can sync the symlinks properly.

[stealthybox]

Does syncthing have a way to follow symlinks for a directory to the actual files so you can keep syncing just the ./bin directory? Alternatively, could you create an archive that follows the links and sync that with a Makefile or something?

[stealthybox]

I'm not sure about is if we need the extracted folders to be hidden.

Agreed. I would think we would not change any folder names. We could store them in a hidden folder underneath the binary dir: ~/bin/.lib/ Or create another location under: "${XDG_DATA_HOME:-"$HOME/.local/share"}" ?

Unrelated, but what about ~/.local/bin ?

ref: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html (I feel like this spec is missing some important things, but it's better than nothing)

[skorokithakis]

@stealthybox yes, Synching can sync symlinks as long as they're relative inside the synced directory. I'd probably end up having bin's folders in a completely separate, dedicated directory, to be honest, as it doesn't make sense to put them all on the PATH.

[stealthybox]

Synching can sync symlinks as long as they're relative inside the synced directory

What about following/resolving the symlinks so synching would sync the file it points to? I was thinking that an option like that would let you keep your "only copy the binaries" use-case if the bin library folder was outside the bin directory or you ignored the library.

+1 though relative symlinks is the way to go so that the bin folder and any other dependent folders are portable.

it doesn't make sense to put them all on the PATH.

Does having a hidden ~/bin/.lib/ folder solve this? PATH would ignore it, and it's just a single inode so it won't affect path lookups negatively. This would make the ~/bin/ directory completely self-contained -- all the symlinks would be valid, even if you moved the directory.

[stealthybox]

I haven't mentioned this, but one of the other reasons I want the folders to be saved is so that I can symlink to the other non-binary files as well.

For instance, if a project ships shell completions, I want to be able to source those files in my .bashrc or zsh ${fpath} -- same thing for manual entries.

Maybe there are even things bin could do to heuristically point these sorts of files out at install/upgrade time, but that's a different feature.

[skorokithakis]

I was thinking that an option like that would let you keep your "only copy the binaries" use-case if the bin library folder was outside the bin directory or you ignored the library.

That's fine, since I sync the bin config file, so I don't have to sync the binaries themselves as well, I guess.

+1 though relative symlinks is the way to go so that the bin folder and any other dependent folders are portable.

Agreed, not sure what other setups as weird as mine exist, but this seems like a sensible thing to do in general.

Does having a hidden ~/bin/.lib/ folder solve this? PATH would ignore it, and it's just a single inode so it won't affect path lookups negatively. This would make the ~/bin/ directory completely self-contained -- all the symlinks would be valid, even if you moved the directory.

I think it would, though I wouldn't call the main directory ~/bin/, probably something like $XDG_CONFIG_HOME/bin/ might be more appropriate, but many people use bin as a directory to store binaries, so you may want to call the directory something like bin-project to make it less conflicty.

For instance, if a project ships shell completions, I want to be able to source those files in my .bashrc or zsh ${fpath} -- same thing for manual entries.

Yeah, I've definitely needed this, and the more I think about this feature, the more I like it. I think that just claiming a directory somewhere under $XDG_CONFIG_HOME and symlinking that to ~/.local/bin/ or somewhere in the PATH (as you described) would be best.