tvm icon indicating copy to clipboard operation
tvm copied to clipboard

[LLVM][RUNTIME] Add optional LLVM ORCJIT runtime executor

Open cbalint13 opened this issue 2 years ago • 1 comments

This PR introduce a new LLVM ORC JIT executor for the runtime. The new ORCJIT may obsolete MCJIT due to better upstream maintenance.


Changes

  • Old MCJIT is default , thus the previous behaviour is unchanged.
  • The new ORCJIT is added as a alternative and optional llvm runtime engine.

Rationale

  • MCJIT is the first implementation in LLVM and looks to be unmaintained by upstream.
  • The new ORCJIT is added and it shares the same flow configuration as the current MCJIT.
  • Targets like RISCV, and possible others too, will not work on MCJIT see some discussions here .
  • The new ORCJIT offers flexibile custom compilation and linkage flow that can be useful for future challenges.

Usage

  • The default is the old MCJIT so no changes in any behaviour of TVM, but being limited it fails on e.g. RISCV:
[23:35:43] src/target/llvm/codegen_llvm.cc:185: Warning: Set native vector bits to be 128 for riscv64
LLVM ERROR: Relocation type not implemented yet!
  • The new ORCJIT can be invoked by llvm -jit=orcjit {...}, this works even on RISCV targets.

This implementation here is backward compatibile and was tested on LLVM={10,11,12,13,14,15,16,17,18}


Halide project also leverage orcjit runtime. Also may relate to PR https://github.com/apache/tvm/pull/14836 and TK #10141 ongoing effort toward RISCV tensorization.

cbalint13 avatar Oct 21 '23 22:10 cbalint13

This is ready for review. Cc: @Lunderberg , @junrushao , @masahi , @vinx13, @tqchen , @ekalda , @lhutton1 , @quic-sanirudh , @kparzysz-quic Cc: @katebern-grovety , @PhilippvK

cbalint13 avatar Oct 23 '23 10:10 cbalint13

@cbalint13 do you mind to rebase on latest main? love to get it in

tqchen avatar Mar 07 '24 17:03 tqchen

@tqchen

@cbalint13 do you mind to rebase on latest main? love to get it in

It is rebased now, glad to see it as being useful. Thank you !

cbalint13 avatar Mar 08 '24 03:03 cbalint13