mlton -codegen llvm works with LLVM 13 but not 15 or 16, and this is not documented

[mcclure]

This page and this page on the wiki state that mlton can be run with LLVM-based codegen. The second page says that "It requires LLVM version 13.0 or greater to be installed.", implying there is no version cap on the LLVM.

For testing, I am on up-to-date Ubuntu 23.10. I apt-get install mlton llvm, giving me "MLton 20210117+dfsg-3" and "Ubuntu LLVM version 16.0.6". I put print "Hello world!\n";; into the file hello.sml. I run mlton hello.sml && ./hello to confirm my mlton is working. Then I run:

$ mlton -codegen llvm hello.sml 

This prints:

The `opt -passname` syntax for the new pass manager is not supported, please use `opt -passes=<pipeline>` (or the `-p` alias for a more concise version).
See https://llvm.org/docs/NewPassManager.html#invoking-opt for more details on the pass pipeline syntax.

MLton 20210117+dfsg-3 raised: Fail: call to system failed with Fail: exit status 1:
opt -mem2reg -O2 -o /tmp/fileZgzw7x.opt.bc /tmp/file2nBVCV.as.bc

To me this implies current mlton is not compatible with llvm-16 due to a change in arguments that happened at some point.

What about other LLVMs?

Ubuntu apt offers llvm-13 through llvm-16. I tried installing llvm-15 (15.0.7), then (because I did not find in the linked docs a way to specify to the MLTon codegen the name of my executables— this would be a nice feature) ran

mkdir bin && ln -s `which llvm-as-15` bin/llvm-as` && ln -s `which opt-15` bin/opt
PATH=$PATH:`pwd`/bin mlton -codegen llvm hello.sml

This also failed, with a different error:

Cannot specify -O# and --passes=/--foo-pass, use -passes='default<O#>,other-pass'
MLton 20210117+dfsg-3 raised: Fail: call to system failed with Fail: exit status 1:
opt -mem2reg -O2 -o /tmp/fileZgzw7x.opt.bc /tmp/file2nBVCV.as.bc

I then tried llvm-13 (13.0.1), and:

mkdir -p bin && ln -s `which llvm-as-13` bin/llvm-as` && ln -s `which opt-13` bin/opt && ln -s `which llc-13` bin/llc
PATH=$PATH:`pwd`/bin mlton -codegen llvm hello.sml

This produced a working, runnable hello.

Conclusion: LLVM 13 works, but LLVM 15 and 16 do not.

Expected behavior

Either mlton should be updated to support llvm-15 and llvm-16, or the wiki should be updated to indicate the maximum supported version.

(Additionally: If Mlton is going to be sensitive to LLVM versions in any way— for example it requires LLVM 13 exactly, or if at some point in future it requires 13 or 16 but does not support 14 or 15— I would expect there to be a specific, documented way to configure mlton -codegen llvm to look for executables at specific paths or with specific prefixes/suffixes, rather than requiring the global opt, llc, llvm-as to be one of the supported LLVM versions. This would improve support on systems with multiple LLVMs installed simultaneously.)

[MatthewFluet]

Short answer to naming tool executables: the -llvm-as <llvm-as>, -llvm-llc <llc>, and -llvm-opt <opt> compile-time arguments can be used to specify the names of tool executables.

In order to avoid linking to the LLVM libraries, we communicate with LLVM via files and the LLVM tools. Unfortunately, that project did not provide forward or backward compatibility with the pass manager syntax for opt, so there seems not to be an easy way of supporting both.

The next release of MLton will use the new pass manager syntax (#491).