Implement internal Ruby activation mechanism

[vinistock]

Motivation

Closes Shopify/ruby-lsp#1776, closes Shopify/ruby-lsp#1784, closes Shopify/ruby-lsp#1794, closes https://github.com/Shopify/ruby-lsp/issues/1216, closes Shopify/ruby-lsp#1762

Currently, we activate the Ruby environment using the user's version manager invoked through a shell. This is necessary because most version managers are shell scripts loaded in config files like ~/.zshrc or ~/.bashrc.

Unfortunately, this has been causing headache because some shell plugins or configurations may accidentally block or interact with our activation script, which leads to difficult to debug issues.

I believe we can provide a better experience if we provide a minimal Ruby environment activation mechanism implemented directly inside TypeScript so that we can avoid interacting with the shell altogether.

Possible drawbacks

While this will fix the issues related to interacting with the shell, it's not free of drawbacks. Certain version managers provide more complex functionality that may not be supported by our internal mechanism.

For example, rtx supports adding extra environment variables in its .rtx.toml file. Other version managers allow users to change the default GEM_HOME where user gems are installed.

Moving forward with this approach means that we might need to add some extra complexity to handle this scenarios or at least make certain things configurable.

Release strategy

We will need to have a long preview release cycle for this one to ensure that activation is working fine for common setups.

Implementation

The bulk of the changes are in Ruby. All other changes are just putting pieces together.

The idea of the activation mechanism is that we

1. Try to determine the Ruby version by looking into common config files (e.g.: .ruby-version, dev.yml, .tools-versions, .rtx.toml)
2. We then try to find a Ruby installation for that version in the directories common used by version managers
3. If we can't find a Ruby version or can't find the directory for the installation, we prompt users to manually select their Ruby binary
4. We then run a script that basically just returns GEM_HOME and GEM_ROOT to us
5. Finally, we modify the environment to include these
   • GEM_HOME -> must point to the user directory for gem installations
   • GEM_ROOT -> must point to the directory for default gem installations
   • GEM_PATH -> a concatenation of GEM_HOME and GEM_ROOT
   • RUBY_VERSION - the Ruby version
   • PATH -> must contain the executable paths for user gems, default gems and for the Ruby bin folder

Automated Tests

Added tests for all different config files we support.

Manual Tests

We need to ensure that Ruby environment activation works properly for chruby, rbenv, rvm, shadowenv, rtx and asdf.

For all of them, the manual tests are analogous:

1. Write the configuration file with a Ruby version (e.g.: .ruby-version)
2. Start the Ruby LSP on that directory
3. Verify that the right Ruby version was picked
4. Verify that the LSP boots properly

[timriley]

@vinistock Just to confirm how I can best test with this using rtx, should I be continuing to use the configuration recommended in https://github.com/Shopify/vscode-ruby-lsp/blob/main/VERSION_MANAGERS.md#rtx?

[vinistock]

@timriley the idea is that you shouldn't need any configuration when using this new mechanism. Don't remove your existing configuration just yet, since we may find bugs and you might need to revert to the stable version of the extension.

The idea is that we now search for your Ruby installation automatically in all paths that are common for version managers. So as long as you have your .tools-versions or .rtx.toml file, we should be able to find it and activate automatically.

Please report back here in the PR if you find that's not the case and thank you for testing it 🙏.

[vinistock]

Note for self: when tools add another gem installation, it usually does not override Gem.user_dir, but instead adds another entry to Gem.path.

We should do something like this:

paths = Gem.path

# There should usually only be the user dir and the default dir. If there's more
# we have to find the other custom entries
if paths.length > 2
  paths.delete(Gem.user_dir)
  paths.delete(Gem.default_dir)

  # Whatever is left are custom directories for installing gems. We can pick the
  # first one
  user_dir = paths.first
end

[Nowaker]

FYI, I switched from release version of Ruby LSP to pre-release 0.6.7 and Ruby version detection fails.

.ruby-version says 3.3.0. Using RVM to switch between versions.

Output:

2024-02-03 14:08:53.454 [info] (worldmodern) Discovered Ruby version 3.3.0

And error message:

So it appears RVM doesn't work on Ruby LSP 0.6.7.

[DuncSmith]

Hi, I'm seeing similar issue to @Nowaker, only I'm using mise.

macos Ruby 3.3.0 ruby-lsp 0.13.4 vscode extension 0.6.7 mise 2024.2.2

I have a .local.mise.toml with

[tools]
ruby = "3.3.0"

I get the same Automatic: Ruby detection failed... message, and in the output 2024-02-05 09:23:27.163 [info] (reponame) Discovered Ruby version 3.3.0

So far the only solution I've found is to use the release version (which works fine with this) or hardcode the path, which is not ideal.

"rubyLsp.rubyExecutablePath": "/Users/<username>/.local/share/mise/installs/ruby/3.3.0/bin/ruby",

[vinistock]

@Nowaker are your rubies installed under ~/.rvm/rubies? And if so, what's the right path to the Ruby executable? Is it ~/.rvm/rubies/3.3.0/bin/ruby?

@DuncSmith what's the purpose of the .local.mise.toml file? When do you use that over .mise.toml? Does it interact with other configuration files in any way? And where do you place it (inside the project or in your home dir)?

[DuncSmith]

@DuncSmith what's the purpose of the .local.mise.toml file? When do you use that over .mise.toml? Does it interact with other configuration files in any way? And where do you place it (inside the project or in your home dir)?

I'd add the local version to my gitignore and use it to override any default config. But I think though this was a bad example on my part, apologies. I do see the same issue with a project specific .mise.toml or without, and fall back to the global config, in my case I have this in ~/.config/mise/config.toml

[Nowaker]

@Nowaker are your rubies installed under ~/.rvm/rubies? And if so, what's the right path to the Ruby executable? Is it ~/.rvm/rubies/3.3.0/bin/ruby?

@vinistock Yes to both. A standard RVM installation.

[vinistock]

@Nowaker And you don't have any other logs after the line you already mentioned? It discovered the right version and then stopped printing?

2024-02-03 14:08:53.454 [info] (worldmodern) Discovered Ruby version 3.3.0

[vinistock]

@DuncSmith I'm not a user of Mise, so I'm going to need more information to get the implementation right.

• In which directory does each configuration file go? Are both .mise.toml and .local.mise.toml intended to be placed inside projects?
• For the global config, that looks shell specific. Is it always under ~/.config/mise/config.toml or is that because you are using fish? Is there a list of the other directories where this global configuration might go for other shells and other operating systems?
• How do the files interact? I assume that the project-specific ones take precedence over the global one, but what's the relationship between .mise.toml and .local.mise.toml?

[Nowaker]

@vinistock Correct, no more logs than what I provided. Happy to provide more information if I'm told where and what exactly to look up. Thanks :)

[vinistock]

@Nowaker if the directories are the default ones, then I suspect it's some permission issue. We were using fs.access with R_OK to check if the directory is readable.

I changed the code to use F_OK, which checks if the directory is visible. Then before trying to boot anything, I also added a check to see if we have permission to execute, since we run exec for the Ruby executable.

Please let me know what happens when you upgrade.

[Nowaker]

@vinistock Aye aye. But how do I check - did you release a new pre-release version to the VS Code Marketplace? As that's how I tested the thing.

[vinistock]

Yup, I just shipped v0.6.10.

[medwards1771]

Hi @vinistock 👋

i'm using the rubyLsp extension in a vscode devcontainer and having trouble getting past the error: Failed to activate Ruby environment. [see screenshot below].

my setup

• vscode v1.87.2
• rubyLsp v0.6.10
• devcontainer details:
  • the container environment does not use a ruby version manager. the value of which ruby while successfully (using rubyLsp v0.5.17) connected to the devcontainer is usr/local/bin/ruby. the devcontainer doesn't install ruby; instead it relies on a docker image with ruby pre-built into it
  • devcontainer.json contents:

{
  "name": "<redacted>",
  "dockerComposeFile": "../docker-compose.yml",
  "service": "devcontainer",
  "runServices": ["postgres", "redis"],
  "workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}",
  "remoteEnv": {
    "LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}",
    "GIT_EDITOR": "code --wait"
  },
  "features": {
    "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
      "version": "latest",
      "enableNonRootDocker": "false",
      "moby": "true"
    },
    "ghcr.io/devcontainers/features/common-utils:2": {
      "installZsh": "true",
      "configureZshAsDefaultShell": "true"
    }
  },
 ...
  "customizations": {
    "vscode": {
      "extensions": [
        "Shopify.ruby-lsp@prerelease"
      ],
      "settings": {
        "terminal.integrated.defaultProfile.linux": "zsh",
      }
    }
  }
}

debugging steps

1. i tried setting the value of rubyLsp.rubyExecutablePath in devcontainer.json's customizations.settings to usr/local/bin/ruby, but this failed with the same error, Failed to activate Ruby environment: Command failed: <see screenshot for full command> /bin/sh: 1: /workspaces/forem/ruby: not found

[screenshot]

[medwards1771]

Hi @vinistock 👋

i'm using the rubyLsp extension in a vscode devcontainer and having trouble getting past the error: Failed to activate Ruby environment. [see screenshot below].

I was able to resolve this by setting "rubyLsp.rubyVersionManager": "auto" in .vscode/settings.json and uninstalling asdf

[vinistock]

The improvements identified with the help of this PR have all been shipped in the stable version of the Ruby LSP extension. In addition to that, all the code has been moved to our monorepo https://github.com/Shopify/ruby-lsp.