Add ability to set the shell

[scottTomaszewski]

What problem are you trying to solve?

In the init_hook I would like to source some scripts containing helper utilities that are written in bash. In devbox run the shell is sh whereas devbox shell the shell is inherited from the caller's shell.

This behavior was confirmed in discord:

We designed it this way initially to make run calls more portable  and consistent. The thinking was that devbox run (especially if used in CI) would use a consistent shell and not be affected by a users specific settings, while devbox shell would use the users shell

which make sense, but if I wanted a consistent shell environment across both devbox shell and devbox run it would be nice to have a way to set that.

More discussion:

• https://discord.com/channels/903306922852245526/1234953700217323541
• https://discord.com/channels/903306922852245526/1251154511746039828

What solution would you like?

Ideally, the solution would be to have support for overriding the default shell for devbox shell and devbox run. For (a contrived) example:

{
  "$schema":  "https://raw.githubusercontent.com/jetify-com/devbox/0.11.0/.schema/devbox.schema.json",
  "packages": ["bash@latest"],
  "shell": {
    "exec": {
      "shell": "/bin/bash --rcfile something.bash",
      "run": "BASH_ENV=something.bash /bin/bash -c"
    },
    "init_hook": [
      "ps -p $$",
      "source .bootstrap.sh"
    ],
    ...

• Splitting out .shell.exec.shell and .shell.exec.run allows for flexibility, but if even without the split my use case is satisfied
• Allowing arguments (not just a shell path) allows for flags to be passed and env vars to be set if necessary
• I understand that this makes for a bit of a mess with inheriting the existing shell. Overriding the shell with this may assume that the --pure flag is automatically set

Alternatives you've considered

• Set the SHELL in the devbox.json env (no effect)
• chsh in the init_hook (requires password or mucking with global filesystem)
• eval bash, bash -l, and other means to starting the desired shell (creates a subshell which doesnt run the devbox run scripts)
• rewriting all our helpers in sh (if I declare bash as a package, I should be able to use it in scripts)
• Prefixing all script lines with bash to force the shell (verbose and problematic with the creation of subshells)

[Lagoja]

I think this is a really good suggestion, and it would help with some shell incompatibility bugs we've seen in the past.

I think I would tweak the proposal to allow the user to specify a shell package they want to install and use as well. For example (not final):

  "shell": {
    "package": "[email protected]"
    "exec": {
      "shell": "bash --rcfile something.bash",
      "run": "BASH_ENV=something.bash bash -c"
    },

Would install Bash 5.1 from nixpkgs and then run shell/commands/services in that shell. This way you could write zsh/bash/fish specific scripts, and have it work across systems regardless of the user's host shell.

[mvanholsteijn]

It is counter-intuitive that the host shell is taken, given that the we expect an isolated, reproducible development environments that runs anywhere

Bash is installed by devbox and is used when running --pure. It would be logical to use this as the default shell for all cases. An override for the default shell would be optional.

  "shell": {
    "package": "[email protected]"
    "cmd": "bin/zsh -c"
 }

Allowing for separate shells for shell, init_hooks and run can become pretty confusing.

[scottTomaszewski]

Allowing for separate shells for shell, init_hooks and run can become pretty confusing.

I agree, but from what I understand shell and run execute with different interactive/login settings which may require different cmd options. For example, --rcfile vs BASH_ENV. I could be wrong.

[mvanholsteijn]

@scottTomaszewski After diving a litter deeper, I suspect that the need for different configurations for run and shell commands is the init_hook commands. These commands need to be sourced in for both the shell and run commands.

Should you be able to reference the init_hook command script in the exec specification?

  "shell": {
    "package": "[email protected]"
    "exec": {
      "shell": "bash --rcfile ${init_hook_path}",
      "run": "BASH_ENV=${init_hook_path} bash -c"
    },

[svallory]

I've wasted a lot of time trying to figure out why some things worked if I ran from a devbox shell but did not when using devbox run :(

Initially, I expected a completely isolated system (even with different $HOME dir), when I noticed it wasn't, the expectation was that things where running in my shell (zsh).

I tailored all the scripts to zsh, but found out that scripts run on sh and --pure shells run on bash. This has been very frustrating journey so far.

Would it really break everything to have a setting on devbox.json that specifies the shell to be used for everything?

[armenr]

+1 for this issue in general.

After all, the value of devbox is the guarantee of a fully-reproducible shell/environment.

We've gone through a lot of pain with this, since different folks have their personal preferences between zsh, fish, and bash on their local development machines.

When they enter the devbox shell, that is our opportunity - as an organization - to enforce our opinion/requirement that "if you want to work with all the stuff in this repo, then you will have to conform to <our_chosen_shell>, since we've standardized all of our scripts on <our_chosen_shell> and all of its capabilities."

Additionally, as discussed above, having the ability to specify that shell FROM the available packages in devbox, and being able to specify even the version of that shell's package...this is all very attractive, valuable, and (IMHO) necessary.

[sascha-coenen]

+1 for adding this feature

having this ability in devbox is without question the highest item on my wishlist. Please, please please, can we have this feature in devbox as soon as humanly possible?

[Nugine]

I'm new to devbox. My first try makes me confused about which shell devbox is using.

My login shell is bash but I use fish in gnome terminal and vscode. devbox shell runs bash because SHELL env variable is /bin/bash. It does not inherit the caller shell (fish).

I cannot find a way to configure this behavior. And finally I find this issue.

[ashishkurian]

@scottTomaszewski Did you manage to fix this?

[scottTomaszewski]

No, my hacky workaround is to have a source .utils/.utilsrc in the init_hook and everything delegates to a justfile that has:

export BASH_ENV := ".utils/.utilsrc"
set shell := ["bash", "-c"]

Im using justfile for everything anyway and devbox is just used for package management... There is probably a better way to do this, but i haven't gotten around to figuring it out

[svallory]

Can't believe this is still open. This issue is what made me abandon devbox :/ Whiich is very sad because appart from this, it's a very cool product. And please, devbox team, understand that it is not about being picky. The amount of familiarity I have with ZSH (in my case) and resources I would be able to reuse simply save me more time then dealing with bash to setup a devbox.

And this is not about shell X is superior. It's just that the cost of adopting devbox when I have to learn bash (a shell I don't use) and port a bunch of stuff is simply too high

[barraponto]

@svallory yeah, it sort of was the reason i left too. but OTOH, it kind of changed my path: maybe i don't need to enforce the shell. maybe all I need is to make the tooling available and leave the user to their own devices.

of course, if a shell script is part of the project, I can make the shell available for that script. though if i'm providing the tools, i can do better than shell script.

[ka-rostom]

Just to follow up (and without any attitude or sass), is there any interest, intent, or activity on jetify team's end to implement this, or nah?

[svallory]

@svallory yeah, it sort of was the reason i left too. but OTOH, it kind of changed my path: maybe i don't need to enforce the shell. maybe all I need is to make the tooling available and leave the user to their own devices.

I actually think the user should have a choice. My case was the opposite of yours. I was forced to do things in bash because that's the shell devbox uses and there's no way around it.

I don't remember exactly what it was, but, for example, you cannot set an environment variable from a subshell. So executing a script in a zsh subshell was not an option and I would have to migrate several scripts to bash, so I simply went to a competitor.

[apgrucza]

@svallory I worked around the fact that you cannot set an environment variable from a subshell by running the below command in the init hook:

eval "$(print_init_env.sh || echo false)"

print_init_env.sh can be written in any shell language you like (using a shebang), but it must output a script that sets the variables, e.g.:

export FOO=foo
export BAR=bar

But I agree it would be so much easier if Devbox allowed the init hook shell to be configured.

[svallory]

@apgrucza that's a clever trick, but I was modularizing the configuration using devbox plugins. I was getting the values from the config, so it would be a huge hassle and potentially a security vulnerability to run eval

[davidmarble]

@svallory Mind sharing what you use instead of devbox? I'm a new user and running into the issues you mention. Feel free to email if you prefer not to share on this issue.

[svallory]

@svallory Mind sharing what you use instead of devbox? I'm a new user and running into the issues you mention. Feel free to email if you prefer not to share on this issue.

I don't mind (I just hope this isn't rude)... I'm using flox.dev, it supports bash, zsh, fish, and tcsh.

[barraponto]

fwiw, i went with flox.dev too. the layering environments feature made me realize all I needed was a single layer with the tools -- leave the actual shell to the user.

it does support environment variables too, btw.

[HiroyukiMakita]

@svallory I worked around the fact that you cannot set an environment variable from a subshell by running the below command in the init hook:

eval "$(print_init_env.sh || echo false)" print_init_env.sh can be written in any shell language you like (using a shebang), but it must output a script that sets the variables, e.g.:

export FOO=foo
export BAR=bar

But I agree it would be so much easier if Devbox allowed the init hook shell to be configured.

Running the following command in the init hook enables the use of environment variables defined in files such as .env located in the project root.

This method works with both devbox shell and devbox run.

    "init_hook": [
      "cd $DEVBOX_PROJECT_ROOT",
      "set -a",
      "[ -f .env ] && . \"$DEVBOX_PROJECT_ROOT/.env\"",
      "set +a"
    ]

By the way, I'm all for adding the ability to change the shell, and I'm looking forward to it too.