urbas
Benchmarking execution
Hi, I've been thinking about rix's idea of using javascript as the execution backend for nix, and I'm unsure whether it will actually benefit the performance.
Logically speaking, a JIT is only useful when there is a lot of repetitive execution that can be lowered to machine code, for faster repetition. Nix however is designed to be a schema description language with only a bit of internal code reuse.
Will the overhead of parsing nix, then converting it to js, then getting v8 to parse the js and execute it be smaller than just executing Nix directly? There is a LOT of lines of code in nixpkgs, so I feel like the generated javascript would be quite a lot for javascript to have much of a performance benefit.
I feel like it should be a priority to get rix to the point where it can execute nixpkgs, and potentially generate the output derivation specifications for a system configuration (purely on the nix language evaluation side). Then it would be possible to benchmark complex real-world configs against each other.
I know one of the biggest hurdles for that is file imports, though I have ideas on how to achieve that.
Anyway, I'm curious about your thoughts on this topic as well
Yes, we need benchmarks. I suspect that there might be enough code reuse in a lot of nix expressions so it might actually pay off.
However, there are many other nice things about using JS:
- profilers
- debuggers
- source code mappers
- huge open source community and industry backing
- actually, v8 can cache compiled JS modules (this should help us avoid a lot of parsing/compiling of a lot of files that don't change over time; might be quite a speed up)
You mentioned transitioning to using Nix's parser for this. However, do you think it might be best to first get rix to a state where it can execute nixpkgs? Because in my opinion it feels really close. The hardest parts as far as I can tell are:
- Implementing more builtins (Just the ones without side effects, things like fetchUrl aren't used in nixpkgs directly)
- Getting imports working (I believe we can just make it a rust function that's attached to v8, which just invokes the imported module and returns the resulting javascript value to the importer)
After that, benchmarking should be viable. What do you think?
Sounds good, let's get that working then. Let me go through the PRs real quick.
is there a burndown list of what builtins are missing for evaluating nixpkgs?
Not quite, but we just assume that all builtins are necessary because nixpkgs is massive, and the full list of unimplemented builtins is in our code (as stubs): https://github.com/urbas/rix/blob/master/nixjs-rt/src/builtins.ts https://github.com/urbas/rix/blob/master/src/tests/builtins.rs
Should be relatively straightforward to implement, feel free to PR implementations in There are some very low hanging fruit if you want to give it a go