prefix-dev
Pixi global to more easily expose npm, gem and friends
Problem description
I previously asked in #2281 about Ruby's gem command: it can install useful programs but these cannot be accessed by default for a pixi global user. This also affects other tools like npm, e.g. today I wanted to use pixi global to install nodejs then use npm to install anthropic's new claude-code (npm install -g @anthropic-ai/claude-code; claude` from here)- this failed as the npm tools are not available in my path -- but works if I use other package managers. When #2281 was closed it was undoubtably for sound technical reasons, however pixi's current limitation still conflicts with the pixi motto: "Package management made easy".
So the feature request is for a future pixi that can better expose executables reliably from npm and gem in the first instance into the path pixi uses for global tools. I know I can manually symlink things (adding the gem path to the user path at least doesn't work due to how it works, so gem executables are better symlinked directly into ~/.pixi/bin). In my mind some command like:
> gem install pandocomatic
> npm npm install -g @anthropic-ai/claude-code
> pixi global expose add claude -e nodejs
> pixi global expose add pandocomatic -e ruby
I, the user know which tool I installed, and pixi know what the hidden path that gem or npm uses is in its env, so then pixi can symlink (or builds a launcher). This is not automatic (although pixi could automate this at a later date), it just makes our life a bit easier. Thoughts?
How would this as opposed to just installing pixi global install gem directly? I'd guess that you would have a version that's coupled to your other dependencies?
Hi, gem at least comes packaged as part of ruby, it is not separate, and I think the same is true of npm as a component of nodejs (I'm not a javascript person). Both gem and npm then install packages for ruby or js respectively. Some of those packages expose executables (like the claude CLI app). When you install e.g. ruby directly (not with pixi), and then install a gem package with an executable that gets added to a system path automatically, so the user only needs to gem install xxx and then can access xxx. I think pip has been disengaged from Python in the condaverse so in this case perhaps installing a pip tool exposes the exe in the pip environment? But that doesn't happen for gemornpm`.
Yes makes sense but why do you want the ruby or nodejs from that specific environment exposed?
Perhaps you have a solution I am not aware of as an end user. I don't particularly care which environment is used on the backend, I want to simplify installing packages that are useful to an end user. If that is done by some background trick that would be fine by me. But there is no separate gem or npm environment (pixi global install npm = Error: Γ Couldn't install npm), it comes as an integral part of the language it was written for.
My ideal scenario:
User would love to use asciidoctor (a great tool for publishing, written in ruby and installable with gem), pixi steps in:
pixi global install ruby
gem install asciidoctor
asciidoctor --help
Ruby is a hard dependency for both gem and asciidoctor. What is your solution with environments to make this straightforward?
Well, I would just install asciidoctor seperately, from conda-forge. And if the ruby/gem versions are the same, it would just link it from the global package cache, and that would not take up any extra disk space :)
Ummmmm, OK asciidoctor wasn't a good example π€£ β but there are many other tools which conda has not packaged separately (I use pandocomatic for managing pandoc, and was trying out claude-code, but I've come across more of these sorts of small tools distributed in this way).
Haha ok yeah, I get it, so the thing is I think is that we do not know the exact location of the gem in this case, though we can figure it out. Also we'd rather have it packaged on conda-forge of course, but I know this is not always the case. Any thoughts @Hofer-Julian :)?
@iandol that is indeed an interesting problem. I expect that we will support that for pypi dependencies at some point, but probably not for npm, gem and co.
What you can do though is to add more exposed executables with pixi global expose add. If those executables are not in the typical directories even nested paths work: https://github.com/prefix-dev/pixi/issues/2350
@Hofer-Julian β first thank you for the work on that issue. I'm probably holding it wrong, but I tried to expose e.g. claude:
Find my executable:
> fd "claude" .pixi/
.pixi/envs/nodejs/lib/node_modules/@anthropic-ai/claude-code/
.pixi/envs/nodejs/bin/claude
It is in .pixi/envs/nodejs/ root, let's confirm it is executable:
> test -x .pixi/envs/nodejs/bin/claude
Now try to expose it:
> pixi g add -e nodejs --expose 'claude=bin/claude'
error: the following required arguments were not provided:
<PACKAGES>...
...expose only works when installing the parent package. Maybe I pass an existing package and pixi will work out it already exists and just exposes the additional tool:
> pixi g add nodejs -e nodejs --expose 'claude=bin/claude'
Error:
Γ Failed to add executables for environment: nodejs
β°ββΆ Couldn't find executable claude in /Users/ian/.pixi/envs/nodejs, found these executables: ["corepack",
"node", "npm", "npx", "corepack", "npm", "npx", "npm", "npx"]
For this to work it seems --expose needs to be decoupled from add somehow, or pixi works out for an existing env to just add the specified "new" tool. Or perhaps there is a command line I just missed to do this.
Expose is decoupled :)
The command is called pixi global expose add as described above
This is still not working:
> pixi global expose add -e nodejs claude=bin/claude
Error:
Γ Failed to add executables for environment: nodejs
β°ββΆ Couldn't find executable claude in /Users/ian/.pixi/envs/nodejs, found these executables: ["corepack",
"node", "npm", "npx", "corepack", "npm", "npx", "npm", "npx"]
Claude is in that folder:
> ls .pixi/envs/nodejs/bin/
ο c_rehash ο corepack ο genbrk ο gencnval ο genrb ο icuexportdata ο makeconv ο npm ο openssl
ο claude ο derb ο gencfu ο gendict ο icu-config ο icuinfo ο node ο npx ο pkgdata
Same for my Ruby tool:
> pixi global expose add -e ruby pandocomatic=share/rubygems/bin/pandocomatic
Error:
Γ Failed to add executables for environment: ruby
β°ββΆ Couldn't find executable pandocomatic in /Users/ian/.pixi/envs/ruby, found these executables: ["bundle",
"bundler", "erb", "gem", "irb", "racc", "rake", "rbs", "rdbg", "rdoc", "ri", "ruby", "syntax_suggest",
"typeprof", "rbs", "irb", "typeprof", "rdbg", "racc", "rdoc", "ri", "erb", "ruby", "syntax_suggest",
"rake", "bundle", "bundler"]
> ls .pixi/envs/ruby/share/rubygems/bin/
ξ do-pandoc.rb ξ pandoc2yaml.rb ο rbs ο rdoc ο ruby ο ruby-lsp-check
ο irb ο pandocomatic ο rdbg ο ri ο ruby-lsp ο ruby-lsp-launcher
Seems like that is a separate issue, we don't consider symlinks when searching for executables. Will open an issue for that.
I think in this use case the best experience you're going to have is with setting the ruby install directory to something global and adding that to your PATH.
I think expose is a useful step forwards once it supports symlink detection. As an enduser it still feels to me like pixi could be doing some more magic to help here. A hard coded folder search (share/rubygems/bin/ for ruby and bin for npm) could allow a user to see which commands are available, like pixi g expose list -e nodejs would give a returned list of core commands and package executables:
pixi global expose list -e nodejs
βοΈ Available executables for environment: nodejs
β°ββΆ "claude", "corepack", "node", "npm", "npx", "corepack", "npm", "npx"
β¦and then I could just specify add with the name, e.g. pixi g expose add -e npm claude as pixi would know where the claude symlink was. Or list could give the relative path that could be copy-pasted if we have to use claude=bin/claude syntax?
Any update on exposing symlinks? I'm running into this issue too.
I just got into this problem when trying to use copilot-language-server. It's not available in conda-forge (yet?), and after npm install -g '@github/copilot-language-server', with npm from pixi, pixi g e add copilot-language-server says it can't find the binary even though its in the env bin dir and it's a symlink. However, after doing pixi global install -e nodejs bash-language-server, which also installs a symlink to the env bin dir, pixi exposes the bash-language-server symlink. Is there something in the bash-language-server package that bypasses pixi checks for symlinks?
