rustler

rustler copied to clipboard

This work is based on https://github.com/rusterlium/html5ever_elixir/pull/28 and other subsequent PRs to that same project.

The idea is to enable the usage of precompiled NIFs and avoid the need to have Rust toolchain installed. This can be beneficial for some projects that are using Rustler.

html5ever_elixir is a good example of project using this feature. But a more basic project was made for the demonstration: https://github.com/philss/rustler_precompilation_example

OK, something went badly wrong with the CI :/ I'm going to check that later today.

Can't this be done as a separate mix plugin?

Can't this be done as a separate mix plugin?

@filmor for sure. Discussing with @hansihe we came to this idea of introducing it to Rustler. But maybe a separated project can be easier to maintain. If you prefer I can go in that direction :)

I do personally think this is something it makes sense to include in rustler upstream, but I am open to being persuaded otherwise.

The main reason why I think it makes sense to include it upstream in Rustler is because I think this is something we should have first class support for. Having first class support should hopefully help encourage downstream libraries to use it.

I agree that it makes sense to have means to hook the publish step, but introducing it like this makes it more difficult to come up with alternative implementations or improvements while keeping rustler_mix stable. There are a lot of design choices in here already that may require a bit of time to settle. There is also a chance of lifting this up to a common mechanism in the whole BEAM world, but that is much trickier IMHO if it's integrated into Rustler itself.

@hansihe @filmor Ideas how we should proceed with this PR?

@filmor @hansihe @evnu I'm going to try this feature in a separated project first. You can take a look here: https://github.com/philss/baked_nifs/pull/1

I can close this PR and re-open in the future if you want. Please let me know.

I can close this PR and re-open in the future if you want.

I think we can keep the PR open and come back to it after you tried it in the other project. If the PR works well in that project, I think we should have a final review and merge.

Since this is something I view as really important for NIF libraries, I do still think that this something we should have first class support for at some point.

In my mind the two modes of using a NIF package are either compiling it on the spot, or downloading a precompiled binary. These two modes are just as valid to me. As such, if we support one of them (compiling a NIF on use), we should also provide good support for the second one (downloading precompiled binaries).

I agree that keeping it in a separate library for now is probably a good idea though, since that can allow it some time to stabilize.

I completely agree with this assessment.

Btw, there is a reason to keep it as a separate library: once precompiled, rustler itself can become an optional dependency. So even if merged inside rustler, it should likely remain a separate project in the rustler repo.

Closing now in favor of re-open again in the future if we decide to include https://github.com/philss/rustler_precompiled into rustler. For now it's working good as a separated library :)