Local remote toolchains don't work with rules_rust

[aaronmondal]

This is due to rules_rust handling compilers and linkers slightly differently than rules_cc. We probably need to fix this upstream.

[adam-singer]

This might be related to the issues I've been seeing with running on osx rbe https://gist.github.com/adam-singer/454091259e457f9db22a1555491afee1

From what I observe SYSROOT is pointing to the rust toolchain which contains what we would expect https://gist.github.com/adam-singer/1f4cff04263bba686d9665eec1777f42#file-gistfile0-txt-L24 bin/ lib/

Looking back at the linker issue from https://gist.github.com/adam-singer/454091259e457f9db22a1555491afee1

/private/tmp/native_link/data-worker-test/content_path-cas/../lib/librustc_driver-ba55eb73d0d2fe7a.dylib' (no such file), '/private/tmp/native_link/data-worker-test/content_path-cas/../lib/librustc_driver-ba55eb73d0d2fe7a.dylib' (no such file), '/usr/local/lib/librustc_driver-ba55eb73d0d2fe7a.dylib' (no such file), '/usr/lib/librustc_driver-ba55eb73d0d2fe7a.dylib' (no such file, not in dyld cache)

Which seems to indicate that the process-wrapper or underlying rustc tried to do some LD_LIBRARY_PATH / DYLD_LIBRARY_PATH path resolution. Not uncommon for linkers to have some "standard" path search.

I think what is happening is the way the process-wrapper passes SYSROOT and somehow sets a root environment of sorts, rustc tries to look back from bin and then to lib with ../lib, making the assumption that rust's core libraries are always the same level as the bin folder.

[aaronmondal]

Potentially relevant is that the latest release from a few hours ago included changes to sysroot.

https://github.com/bazelbuild/rules_rust/releases/tag/0.33.0

The darwin issue seems indeed very similar to the issues on linux. Enabling the --incompatible_enable_cc_toolchain_resultion triggers this error: https://github.com/serde-rs/serde/issues/2322 and breaks a bunch of macro resolutions that point to toolchain misconfiguration or some issue with the dynamic linker and/or glibc: https://gist.github.com/aaronmondal/a2f8caf3c20cfec0dc986057e5e2020f

[adam-singer]

Filed a ticket with rules_rust https://github.com/bazelbuild/rules_rust/issues/2331, maybe there is an existing work around, otherwise we can complete the patch. Hooking the ld lib path usually isn't ideal, rules_rust has so many flags and settings, wasn't sure would be the right one to flip.