ORCv2 JIT

[Vindaar]

Hey!

I've been curious about playing around with nlvm and in particular your JIT branch ever since you opened that PR. Took a while... Thanks a lot for this project for sure!

Finally, looked into this and I stumbled pretty hard initially. Couldn't get the LLVM 9 branch to work (later figured out I just had to apply this patch: https://www.mail-archive.com/[email protected]/msg08412.html ) and learned that the API used on that branch isn't available on LLVM 13 anymore. So I decided to try to wrap the new API and managed to get it to work over the weekend. :partying_face:

So far the code is essentially achieving the same as in #18. I started from a rebase of that branch onto the current master.

I commented a bit more than usual on the lljit.nim code, as it was a bit hard to figure out. The documentation of the current C API is sorely lacking still. Most helpful were the examples (ORCv2C in here https://github.com/llvm/llvm-project/tree/main/llvm/examples/OrcV2Examples). In particular assigning a "search generator" for the symbols in the host process was hard to figure out.

I'm opening this PR especially to ask for a bit of guidance. Locally I'm playing around with an attempt to somehow JIT smaller Nim snippets (without having to recompile system etc. each time). For now I'm very much confused about how to best structure such code though. I have some ideas, but you probably thought about this already and have a much better understanding of what's going on in the compiler. So any input would be greatly appreciated!

[arnetheduck]

Ooh! This is great stuff.

My plan was more or less as follows:

• get it to work in any way, shape or form, with simple snippets that don't have library requirements
• add library loading - this is a bit more tricky but still feasible
• enable running the testament test suite using the jit and start tracking success rate - I suspect one can quite easily get to something like an 60-80% pass rate here - this is the step that stands between keeping it a PR and merging it.

at this point, there's a working prototype that effectively replaces nlvm -r. Looks like we found the same orc examples/tutorials to base it off :)

In terms of code structure, I think you're on the right path. Nim doesn't have much of a a module system to speak of, so it doesn't matter as much as one would think - I've intentionally kept nlvm, uh, simple in this regard - at some point, this might change, but besides keeping the glue code that plugs orc into the rest separate, there's really not much to it: one part generates bytecode, the others either turn it into machine code and link or runs it directly.

Once the code is at this point, I'd go ahead and merge it - this is where the fun starts, and there are several ways to move from there:

• replace the nim vm with orc - this requires some upstream changes to make the vm a bit more pluggable - the project uses a custom branch of nim already, and the work can happen there.
• replace incremental compilation by storing and reusing the llvm bytecode
• run the optimizer on bytecode as part of execution, then store the result
• etc

[arnetheduck]

I commented a bit more than usual

yeah, looking at the code with fresh eyes is an excellent opportunity to write down these lessons - if you have any more of that, I'm happy to add them to the codebase.

[Vindaar]

Thanks a lot for your input!

I'm currently finishing up a bunch of stuff here and there. Once those are out of the way, I'll get back to you.

[arnetheduck]

@Vindaar are you planning to continue this work?

[arnetheduck]

superseded by #45