Rust Nightly rust-analyzer RUST_SRC_PATH Wrapper

[garrett-hopper]

I previously used Rust's stable channel along with rust-analyzer from nixpkgs. This provided a wrapper around rust-analyzer which set RUST_SRC_PATH.

I've recently switched to rustChannels.nightly.rust which provides and conflicts with rust-analyzer. This version, however, doesn't have RUST_SRC_PATH set in a wrapper which seems to break most completions in lsp-mode.

I believe I can just set RUST_SRC_PATH myself to fix this, though it seems to me that this overlay should be updated to wrap rust-analyzer in the same way the nixpkgs derivation wrapped it.

[garrett-hopper]

For what it's worth, this issues also happens when I switch back to the stable channel and use rust-analyzer-unwrapped instead.

I've so far been unsuccessful getting this to work by setting RUST_SRC_PATH manually. lsp-mode still doesn't have standard library completions.

[kyren]

You may have already tried this, but in that case this is for anybody else who comes across this issue. I think the previous behavior of the wrapper can be emulated by putting something like this in your shell.nix file:

shellHook = ''
  export RUST_SRC_PATH="${rustChannel.rust-src}/lib/rustlib/src/rust/src"
'';

(Note the trailing part after rustChannel.rust-src, this is apparently the directory that RUST_SRC_PATH should be set to from looking at the rust-analyzer source, and it matches the contents of nixpkgs.rustPackages.rustPlatform.rustcSrc)

It's hard to test because like you my std completions are still not working, but this does get rid of the error from rust-analyzer about correctly setting RUST_SRC_PATH, so presumably that part worked?

[tel]

I've also been trying to figure this out. I don't see the RUST_SRC_PATH error (maybe getting swallowed) but setting it as above doesn't fix the lack of autocomplete.

[garrett-hopper]

RUST_SRC_PATH="${latest.rustChannels.nightly.rust-src}/lib/rustlib/src/rust/src";

Gives /nix/store/cby7s6wpw4dfnzffrm9x4jh8d1jhhib3-rust-src-1.48.0-nightly-2020-08-28-d006f5734/lib/rustlib/src/rust/src which doesn't exist. lib/rustlib/src/rust exists with a Cargo.lock file and a library directory, but there is no src directory in there.

I tried a few different variations on that path, but I was never able to get autocomplete working.

[kyren]

Ah, I think the difference might be that I'm using rust stable, so for me the path is /nix/store/yzyjsq1l2w9g1d8hl25mnbswdw9j1sm4-rust-src-1.46.0-2020-08-24-04488afe3/lib/rustlib/src/rust/src, which definitely exists, and contains:

build_helper  libcore	      libpanic_unwind  libprofiler_builtins  libterm  libunwind  tools
liballoc      libpanic_abort  libproc_macro    libstd		     libtest  stdarch

I still can't get autocomplete working though, but I think my autocomplete problems are larger than this because autocomplete does not work for me in general. Currently, std:: completes with the contents of std:: (any, arch, array, asm, etc...) but any submodule (std::any::, etc...) completes with only self and super, and similar patterns seem to hold for other crates.

This might be unrelated to nixpkgs-mozilla entirely. It definitely appeared the moment I started using the rust-analyzer from nixpkgs-mozilla, but it might just be down to a version change in rust-analyzer or something else.

[tel]

It might not be nixpkgs-mozilla, but I can confirm that I've got autocomplete working through lsp-mode in Python, it's just rust-analyzer and rls that are broken. I'll eventually try doing an isolated test with rustup, too, but I'd much prefer to stay in Nix.

Autocomplete also works fine for other crates, just not std.

[acowley]

I've been grappling with this, too as I didn't turn up this Issue in my searches until now. ~~It seems like some of the needed source files are missing, but I'm not clear on which component is at fault.~~

ETA: Everything is there, but RA isn't picking it up. Feels like a sysroot problem, but rustc --print sysroot looks good, and using RUST_SRC_PATH instead doesn't help.

[acowley]

I think the problem is the use of symlinks in the nix store. After spending too much time on this, I was able to make std work for rust-analyzer by creating a dumb little overlay that I stick on top of everything else that just copies the symlink-resolved rust-src files into its $out path, and creates a wrapper script for rust-analyzer that sets $RUST_SRC_PATH to itself to avoid further fuss. There are several ways to accomplish the same goal, but I think it's the symlinks that are the problem.

Hypothesis: possibly the issue is that rust-analyzer checks the full paths of individual files against lists of directories. We essentially end up with the symlink forest in the list of directories to consider, but then the individual files, once some bit of code has chased symlinks, do not have paths that match those directories.

[tel]

Do you think this is a reasonable ticket to send over to rust-analyzer? It might not be a priority of theirs, but handling symlinks when searching for source files seems like a valuable addition. It'd be useful to create a minimal replication that doesn't rely on Nix as I would bet they don't care about Nix specifically.

[TLATER]

It looks like this isn't the only place rust-analyzer struggles with symlinks: rust-analyzer/rust-analyzer#3691, rust-analyzer/rust-analyzer#717.

I think it's worth adding!

[dfoxfranke]

It seems that as of 1.47 going stable, the correct RUST_SRC_PATH setting is ${rustChannel.rust-src}/lib/rustlib/src/rust/library. Setting this prevents rust-analyzer from complaining at VSCode startup that it's incorrectly set, but it does not make my completions work. OTOH everything works fine if my Rust environment comes from nixpkgs.rustup rather than nixpkgs-mozilla, and I don't have to set RUST_SRC_PATH manually. This is a bummer because I really prefer to keep my dev toolchain managed from within nix, so I hope someone gets to the bottom of this.

[acowley]

@dfoxfranke did you try the suggestion in my comment above? I’ve been using that with working completions.

[TLATER]

@acowley would be nice if you could share your overlay, I also resorted to just using rustup because it's simpler :)

[acowley]

@TLATER Here's part of a shell.nix for a current project that uses niv:

chan-specs = { date = "2020-09-23"; channel = "nightly"; };
ra-wrapper = self: super: {
  rust-analyzerw = pkgs-host.runCommandNoCC "rust-analyzer-1999" {
    pname = "rust-analyzer";
    version = "1999";
    nativeBuildInputs = [ pkgs-host.makeWrapper ];
  } ''
    mkdir -p $out/bin
    cp -rL ${(super.rustChannelOf chan-specs).rust-src}/lib/rustlib/src/rust/library $out
    makeWrapper ${(super.rustChannelOf chan-specs).rust-analyzer-preview}/bin/rust-analyzer $out/bin/r
ust-analyzerw --set RUST_SRC_PATH "$out/library"
  '';
};
pkgs = import sources.nixpkgs {
  overlays = [ moz_overlay rust-src-overlay ra-wrapper ];
};

I did this so I could add rust-analyzerw to my buildInputs and be sure I was getting my wrapped analyzer, but I was really just trying to get myself unstuck rather than packaging things properly. I don't know if there's will to fix things here or try to get RA to handle symlinks differently. I guess a third option is for me to stick this overlay that fixes things somewhere on GitHub so people could use it while the decision of whether to fix here or upstream gets hashed out by the maintainers. I'm open to suggestions.

[dfoxfranke]

I haven't tested the fix yet, but I think I've identified the bug in RA. See https://github.com/rust-analyzer/rust-analyzer/issues/3691#issuecomment-708699545.

[dfoxfranke]

https://github.com/rust-analyzer/rust-analyzer/pull/6246 fixes the symlink problem for me. It remains necessary either to overlay rust-src-overlay.nix or to include

  shellHook = ''
    export RUST_SRC_PATH="${rustChannel.rust-src}/lib/rustlib/src/rust/library"
  '';

in the derivation for my VSCode environment. But this much seems like expected behavior to me, not a bug.

[deifactor]

@dfoxfranke Thanks a ton! What's the nicest way to use a version of rust-analyzer with that fix from within the overlay? When I run rust-analyzer --version, I get 0d03fe6, which looks like it's still a month old, even though I'm running on nightly. My rustc is from yesterday, so that's up-to-date.

[dfoxfranke]

@deifactor I checked out and built https://github.com/rust-analyzer/rust-analyzer and then changed the symlink $HOME/.config/Code/User/globalStorage/matklad.rust-analyzer/rust-analyzer-linux to point to the binary I compiled.

Make sure you build with --release. The debug build is an order of magnitude slower.

[deifactor]

Aah, I was hoping for a more Nix-y solution. I can just copy acowley's trick until this makes its way in.

[NickHu]

A simpler solution if you are content with rust stable is to just use the version of rust-analyzer from nixpkgs like so:

mkShell {
  buildInputs = [
    rust-analyzer # the one from Mozilla is slightly broken https://github.com/mozilla/nixpkgs-mozilla/issues/238
    (moz_nixpkgs.latest.rustChannels.stable.rust.override {
      # extensions = ["rust-src"]; # unneeded because we're using rust-analyzer from nixpkgs
    })
  ];
}

It's crucial that rust-analyzer appears before rust, because the order of buildInputs determines the PATH environment variable.

Then completions and goto-def etc. all seem to work for me (technically you might get out of sync, but I think rust-analyzer from nixpkgs is supposed to track rust stable so it's probably close enough):

❯ rust-analyzer --version                                                                                                  nix-shell 
rust-analyzer 2020-10-12
❯ rustc --version                                                                                                          nix-shell 
rustc 1.47.0 (18bf6b4f0 2020-10-07)

[maisiliym]

  # The packages available usually are:
  #   cargo, rust-analysis, rust-docs, rust-src, rust-std, rustc, and
  #   rust, which aggregates them in one package.

That doesnt seem to be true anymore. Li is unable to find rust-src in the rust 'package'.

[drozdziak1]

has anything changed for this issue? I'd love to be able to use rust-analyzer from nixpkgs-mozilla

[maisiliym]

@drozdziak1, there isn't much point in using a nix-store rust-src; being read-only, it breaks analyzer.

[dfoxfranke]

@maisiliym Breaks it how? It's been working fine for me.

[paholg]

I've stumbled across this issue because I've been having issues with rust-analyzer in Nixos. @dfoxfranke it's broken because rust-analyzer tries to create Cargo.lock, but cannot inside of /nix/store:

LSP :: rust-analyzer failed to load workspace: Failed to read Cargo metadata from Cargo.toml file /nix/store/wfv522w143y3yxx8ap3fkpjk9cnh1w3a-rust-lib-src/core/Cargo.toml, cargo 1.50.0 (f04e7fab7 2021-02-04): Failed to run `cargo metadata --manifest-path /nix/store/wfv522w143y3yxx8ap3fkpjk9cnh1w3a-rust-lib-src/core/Cargo.toml` in `/nix/store/wfv522w143y3yxx8ap3fkpjk9cnh1w3a-rust-lib-src/core`: `cargo metadata` exited with an error:     Updating crates.io index
error: failed to write /nix/store/wfv522w143y3yxx8ap3fkpjk9cnh1w3a-rust-lib-src/core/Cargo.lock

Caused by:
  failed to open: /nix/store/wfv522w143y3yxx8ap3fkpjk9cnh1w3a-rust-lib-src/core/Cargo.lock

Caused by:
  Read-only file system (os error 30)

That said, this thread has helped me to get it working. I needed to set in my .profile (it was in my .zshrc):

export RUST_SRC_PATH="$(rustc --print sysroot)/lib/rustlib/src/rust/src"

which sets it to ~/.rustup/toolchains/...