asdf

asdf copied to clipboard

Is there an official list of what shells asdf supports? Is there any chance for dash/sh to be supported, or is it only bash and korn shells?

https://github.com/asdf-vm/asdf/blob/e79016add33720fbb039ec4e98c5959624704bbb/asdf.sh#L1-L4

Seems to be a partial list of support, but fish is also supported since there's an asdf.fish file.

Currently when run under sh you get an error like: /home/my_home/.asdf/asdf.sh: 8: /home/my_home/.asdf/asdf.sh: Bad substitution

My use case is that I help maintain the Elixir Language Server which uses a sh script to launch the server, but since many users use asdf to manage their elixir installations we recently started sourcing asdf inside that script, but this doesn't always work because /bin/sh might be actual sh (or dash) instead of bash.

Of course the effort involved in changing asdf to be sh-compatible may be large which might make it not worth the effort. So my goal with this issue is to definitively find out if sh/dash is meant to be supported, and if not to have an official list of shells (even if it's not exhaustive) that are supported.

Is there an official list of what shells asdf supports

There is not, though I would like to add this to the documentation. At this stage I would say we explicitly support:

• bash
• fish
• zsh

Though it may be fully compatible with others, we aren't running e2e tests for all shells.

Feel free to correct my understanding, but with this shell history:

❓ sh (bourne) -> ✔️ bash (bourne again) -> ✔️ zsh
               -> ❓ ash                  -> ❓ dash (current /bin/sh on debian-based unix)
               -> ❓ ksh
✔️fish
❌powershell   -> ❓ powershell core

the different base of dash from bash makes me think it is not supported at this time.

@asdf-vm/core

• Could we create a definitive list of the current supported shells with GH Action e2e test cases?
• Are there longer-term plans to be compatible with other shells? Powershell Core has also been requested in #576
• Should we rename the *.sh files to *.bash to better communicate which shells each script is meant to support?
• If we desire to support more shells, can we have a discussion about this project's codebase and how we would support this? (currently we switch case depending on the shell)
  • how would plugins work as most are built with bash as the executing shell?

• Could we create a definitive list of the current supported shells with GH Action e2e test cases?

I don't know if that is feasible or not, but if it is we definitely should. The other day I was reading about the different Bash versions and realized there are significant features that not all versions support. We should know which versions of Bash we support.

• Are there longer-term plans to be compatible with other shells? Powershell Core has also been requested in #576

I guess to me this depends on how much work it is. If it isn't too terrible to support and we can verify everything we need in our builds I'm all for it.

• Should we rename the *.sh files to *.bash to better communicate which shells each script is meant to support?

I was actually thinking about adding back the .sh extension to the files that lack them. Some of our source files now contain magic comments instead of a file extension, and I'd like to avoid magic comments if possible or at least have both the extension and the magic comments. I keep magic comments turned off in Vim because of past security issues.

• If we desire to support more shells, can we have a discussion about this project's codebase and how we would support this? (currently we switch case depending on the shell) how would plugins work as most are built with bash as the executing shell?

Good questions. I don't know. I like to be able to write once and run everywhere as much as possible, and I don't see any issues doing that with the current state of the codebase. Looking at our docs, they only say the custom command* files must be bash. All the other scripts only need to be executable files, so it's possible for them to be anything.

So work we can do to get to a better known state:

• Add .sh extensions back to all source files that lack them and remove magic comments.
• Be explicit in documentation about specific shells we support. At this time Bash, Fish, ZSH.
• Determine the specific versions of each shell we are compatible with (certainly more work than the above, but still should be a goal of ours).

@Stratus3D should we open issues to track that we want to make these changes?

@axelson I'm sorry we don't have a better answer than "Not at this time". It looks like we have some areas to improve before we are in a state to start looking at adding this support. Will keep this thread open should you have other comments or ideas to help us progress, thanks.

@axelson I'm sorry we don't have a better answer than "Not at this time". It looks like we have some areas to improve before we are in a state to start looking at adding this support. Will keep this thread open should you have other comments or ideas to help us progress, thanks.

Thanks for looking into it! Just understanding the state of things is good enough for me right now. I really appreciate all the hard work put into asdf, it is an invaluable tool for me :)

@jthegedus so for the file extension, should we go with .sh or .bash? We would have to change a few more files if we go with the .bash extension, but the added clarity may be worth it.

It depends on what the intent is.

For each file, are we saying .bash because the script requires Bash features to execute? .sh is nondescript given the confusion with bourne vs dash as the OP pointed out, so I'm not sure that would be useful either. I guess if we went with .sh we would require users to know which shell their system uses with /bin/sh, which isn't unreasonable.

In the case raised with the OP, should our goal here be to remove all bash specific features to serve a larger number of shells?

I think these are our options:

1. Support more shells (ash/dash/other) use .sh, remove Bash specific features from the codebase and require users to know their /bin/sh shell.
2. Require Bash shell on the system and rename all files to .bash
3. Rename files that require Bash to .bash and the rest to .sh. Then re-assess the other two options.

So, as far as I know, we do require Bash to be present. The asdf executable in bin/asdf has a shebang line that is specifying bash - #!/usr/bin/env bash. So right now bash itself is a hard requirement, regardless of what shell the user actually uses at the terminal. I think it's good we don't use bin/sh as it is often aliased to other shells as @axelson mentioned.

I'm fairly certain the asdf code would not run properly under Zshell or Dash, but I haven't tried to run it.

I think it's hard enough to write shell programs that work consistently across operating systems in Bash. Making the code work across multiple shells sounds like it would make the code extremely difficult to get right, but I may be wrong.

Since all of the files sourced by bin/asdf are executed by Bash, I think it would be best to give them all the extension .bash for now to make it clear they've been written with the assumption that they will be run by Bash.

For other shells, supporting them could be as simple as adding another asdf.* to the repository. We already have asdf.sh (Bash) and asdf.fish (Fish). It all depends on what is meant by "support". While users may want to use another shell like Zsh, Fish, or Dash (and have nice autocomplete available), having Bash installed shouldn't be too much of a hurdle for most users. OSX ships with Bash and Zsh, and I'm not aware of any Linux distros that lack Bash.

I think you made the right choice to use #!/usr/bin/env bash in ElixirLS @axelson https://github.com/elixir-lsp/elixir-ls/pull/118

I think you made the right choice to use #!/usr/bin/env bash in ElixirLS @axelson elixir-lsp/elixir-ls#118

Thanks!

@Stratus3D I agree 100% with your assessment.

If you're willing to hardcode the path to asdf in the bootstrap script, this is feasible. However, I'm not aware of a way to determine the script location of a sourced script that is POSIX compliant.

I haven't hit any issues yet using a custom POSIX compliant bootstrap, but my use of asdf so far has been installing python and asdf reshim.

I noticed husky (js git hook tool) hits this issue when you manually switch the /bin/sh implementation with sudo ln -sf /bin/dash /var/select/sh.

This could become an issue for future macOS versions, as they have plans for eventually replacing (their ancient) bash in posix mode for /bin/sh with dash.

I haven't hit any issues yet using a custom POSIX compliant bootstrap

Could you share it?

I'm not aware of a POSIX way to get the location of the current script, so this assumes the .asdf folder is in $HOME/.asdf:

asdf.posix.sh

Lacking sh support makes asdf very hard to use in some places (like in CI environments or in Docker containers), it would be really great if it were supported!

Ahoj! I understand that finding ASDF dir is difficult on POSIX. Considering this can be solved by setting the ASDF_DIR variable, can we have some sh compatibility at least for running the tools installed - a minimal scope? For instance [[ is not the most compatible command and can be replaced easily with a dash friendly alternative.

@Stratus3D > The other day I was reading about the different Bash versions and realized there are significant features that not all versions support.

Yes, I'm running into that now trying to do some performance optimizations. For example, bash v4 and on can internally convert case:

bash-5.2$ foo="hEllO"
bash-5.2$ echo ${foo^^}
HELLO

zsh also does this via expansion modifiers (although I'm not sure what version they became available in); also IIRC they borrowed it from tcsh if that was ever something that people wanted to support:

echo ${foo:u}
HELLO

This avoids a call to tr '[:lower:]' '[:upper:]', which dramatically speeds it up:

❯ time (echo $foo | tr '[:lower:]' '[:upper:]') >/dev/null
( echo $foo | tr '[:lower:]' '[:upper:]'; ) > /dev/null  0.00s user 0.00s system 75% cpu 0.006 total

❯ time (echo ${foo:u}) >/dev/null
( echo ${foo:u}; ) > /dev/null  0.00s user 0.00s system 41% cpu 0.001 total

I'm not completely opposed to supporting other shells, but given that for interactivity bash and zsh dominate, I'd rather see their performance maximized by using their built-ins. The shell could be determined with ps -p$$, and any necessary differences could then be branched. There are also ways to overcome some of the obvious problems: e.g. zsh using 1-indexed arrays and bash using 0-indexed can be solved by always using the offset: ${array[@]:offset:length}.

Just an update, PR #1480 currently fixes all issues with POSIX Shell / Dash support. Just manually set ASDF_DIR (since it is impossible to infer the value of ASDF_DIR with those shells), and things should Just Work. I already tested it on all supported shells and it works flawlessly - so if anyone is waiting on this feel free to checkout that branch.

Please test the pre-release code on master with asdf update --head. As noted by @hyperupcall there are changes to the setup instructions for these other Shell support, so check the documentation at https://asdf-vm.com/guide/getting-started.html#_3-install-asdf under the "POSIX" sections.

@stephanGarland for source files with the .bash file extension we can absolutely use Bash built-ins to make things faster. However, for files currently in this codebase with a shebang or extension that just specifies sh, it needs to be POSIX compliant and thus cannot use any Bash-specific features. The good news is most of the really expensive code is contained in files that are only ever executed by Bash (have the .bash file extension) and thus we can use whatever Bash features make the code fastest. Hopefully that clarifies things.