Compile Rust Language Server into WASM to provide IntelliSense and completions

[skyne98]

Partially realted to #188

Hey there, folk! Should we try to compile Rust Language Server into WASM to run it in the playground to provide, at least, the intellisence and completions?

I don't really know how closely it is tied to the rust compiler itself, which cannot be, as far as I know, compiled for the web at the moment. I think we really need someone with a good pack of knowledge here 😄

Performance might be an issue, but it should be possible to, in theory, run the RLS in a separate WebWorker. Maybe even compile it to use multiple of them using Emscripten and enabling SharedArrayBuffer support?

[shepmaster]

Should we try to compile Rust Language Server into WASM to run it in the playground

I'm about 99.99% certain that this will simply not be feasible. At a bare minimum, the compiled version of the RLS that I have is 20 MiB. That is a really hefty download to require of anyone, especially those on metered connections.

I also believe that you'd have to get a huge amount of the compiler to target WASM, which includes a large number of dependencies.

I will not be attempting this, but if someone wishes to take on this Herculean task and is able to show a decent proof-of-concept, of course we'd be interested.

[skyne98]

@shepmaster, I think that enabling RLS assistance should be an optional feature, which could be enabled by pressing a button "Run RLS" or something which will download it and run in a worker. This way users will have to download it only when they want to. Also, 20 MiB is not too big of a size for a browser-integrated semi-environment, which will have to be downloaded once.

About dependencies -- even though I have no idea about the parser and analyzer child packages, theoretically they shouldn't require any "native" code or syscalls.

[shepmaster]

which will have to be downloaded once

It will have be be downloaded every 6 weeks for stable, every few weeks for beta, every day for nightly. If you use all of those at the first time, you'd get 60 MB.

But sure, if it's feasible, then it can be optional, that's not much of a problem. The Ace editor is already an optional download.

[skyne98]

@shepmaster, I can totally understand your scepticism, but it would be super nice to have a snippet sharing full fledged editor for rust. It will allow new users to experiment with rust without needing to install it or do it blindly in the editor. Also, it will allow experienced developers to write and share snippets much faster.

[DJMcNab]

Related: https://internals.rust-lang.org/t/2019-strategy-for-rustc-and-the-rls/8361/62. This is the discussion about getting https://github.com/rust-analyzer/rust-analyzer working in the playground.

[vultix]

There's a pretty cool working example of this here: https://github.com/vultix/rust-analyzer-wasm

To try it out:

• Clone the project
• Run cd www & npm install
• Run npm start
• Open http://0.0.0.0:8080/ in your browser

[shepmaster]

Thank you for your work!

The file is 7 MiB:

% du -h wasm_demo.wasm
7.0M	wasm_demo.wasm

The current playground is about .25 - 0.3 MiB of transferred data. This would be ≈ 23x that size; not ideal. If used, it would need to be opt-in.

[vultix]

@shepmaster I feel I should clarify, the work is not mine! All of the work is from @matklad. I merely branched off his work and pre-compiled the wasm file to save time.

As for the 7 MIB size, we could easily disable the rust analyzer features on mobile if saving data is a priority.

[skyne98]

Also, why not just implement it as a setting that is first decided by a popup for the user? 7 MIB, in my opinion, is not too much for a full-fledged experience, especially when it's opt-in.

[shepmaster]

why not just implement it as a setting

Did “it would need to be opt-in” suggest something other than a setting?

first decided by a popup

The primary target audience of the playground are those who want to experiment with Rust and don’t want to answer a myriad of questions. Perhaps they can’t answer them yet because of a lack of knowledge. Any such interaction needs to be carefully thought through; “just add a popup” isn’t sufficient to me.

[HKalbasi]

@shepmaster is something like this (https://silly-einstein-6e6848.netlify.app/) good? It doesn't download rust analyzer by default, But monaco is heavier than ace (my monaco build in that link is 2.6MB, 620KB gzipped but current playground is 1.0MB, 380KB gzipped) I think monaco pay for itself and this 240KB is tolerable in 2021. If it is OK and there is no problem with monaco and/or intellisense, I am volunteer to do it.

This is the path in my mind:

• Replace ace with monaco
• Make rust-analyzer-wasm usable. Currently it doesn't have support for std, so it is more annoying than useful.
• In parallel to above step, we can use other monaco features, such as showing compile errors inside code
• Add the inventory concept with rust-analyzer as the only item, with service worker
• Enable more features of rust analyzer
• (far future) Add other components like offline rustc-wasm, ... to inventory

If there is no concern (I can see some possible and valid ones) I can work on this.

[shepmaster]

As mentioned here and in other places, I'm not opposed to such changes, I just want them to be worth their effort.

• Replace ace with monaco

#450 has discussion around switching to Monaco; please make sure to read through there for various points. TL;DR: I don't currently see a benefit of replacing Ace.

That being said, I could see an argument for adding Monaco in addition to Ace. Since we already allow people to select between the simple and Ace editor, there could be a third option as well.

Even in a world where we eventually remove Ace, I'd want to have both editors available concurrently for a while.

• Make rust-analyzer-wasm usable

This seems like a non-trivial amount of work and it's unclear what the overall scope would be. Do you have any roadmap for what exactly needs to be done?

What are the benefits to using Monaco before rust-analyzer is usable?

• it doesn't have support for std

Recall that the playground also has ~250 crates that would need to be included to be maximally useful.

• other monaco features, such as showing compile errors inside code

Sure; this would require changing the data sent between the compiler/backend/frontend to use the compiler's JSON output and updating all the relevant places and not breaking existing exposed APIs.

I would 100% prevent this from being the only error reporting format. Many people are already too confused when the editor splits a rustc error across 3 lines and it's hard to realize that the separate colored squiggles are related to each other.

• more features of rust analyzer

This is a bit nebulous — which features would you propose be enabled to start with and which later?

• like offline rustc-wasm

Is there any concrete progress in such a tool? My understanding has always been that the rustc toolchain is hundreds of megabytes and that moving it to Wasm would simply not be useful.

[HKalbasi]

That being said, I could see an argument for adding Monaco in addition to Ace. Since we already allow people to select between the simple and Ace editor, there could be a third option as well.

That is fair. I see no benefit in removing Ace.

This seems like a non-trivial amount of work and it's unclear what the overall scope would be.

I see current features + std support as usable.

Do you have any roadmap for what exactly needs to be done?

I added a subset of std (methods of option and iterator) to my demo. My method is not perfect (and I will look for betters), but I see adding std as completely feasible.

Recall that the playground also has ~250 crates that would need to be included to be maximally useful.

It will silently ignore them (if they are not macro) and replace them with {unknown} so I think we don't need support for them (for start).

This is a bit nebulous — which features would you propose be enabled to start with and which later?

Features that are currently work + std support, i.e. hover, auto complete (suggestion and function signature), jump to definition and (limited) rust analyzer diagnostics, in their current state. Everything else would be in later category, specially code lenses (gray things, like type annotations or Run code above main). I think current state compete to what RLS was when this issue started.

What are the benefits to using Monaco before rust-analyzer is usable?

Monaco, rust-analyzer and service worker/inventory are three distinct units of work. They can be done and reviewed separately. And they are independently useful (Monaco has slightly more features (like multi cursor) so it is valid as additional option and service worker helps loading time in general.) So I see benefit in implementing them in separate PRs. Is it bad?

[HKalbasi]

@shepmaster std is now supported in the original https://ra-wasm.netlify.app/ so ra-wasm is now usable IMO. If you review monaco PR I can continue to work on this.