As the title says, the latest Nuget package for libllvm is missing C API symbols.

joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ llvm-nm-6.0 5.0.1/runtimes/linux-x64/native/libLLVM.so | grep LLVMModuleCreateWithName
0000000000838d20 T LLVMModuleCreateWithName
0000000000838db0 T LLVMModuleCreateWithNameInContext
joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ llvm-nm-6.0 6.0.0-beta1/runtimes/linux-x64/native/libLLVM.so | grep LLVMModuleCreateWithName
joao@joao-HP:~/dev/dotnet/corert/packages/libllvm$ 

This breaks CoreRT under Linux.

Thanks @tritao. We'll get this squared away this week, I also noticed a similar break in CoreRT. FYI @morganbr

@mjsabby is there anything we can do to help fix this up?

I'm not sure how helpful it would be to anyone, but I'm working around this in corert right now by building my own libLLVM and patching it in (since corert unpacks packages and links the native objects directly).

On OSX

Clone the llvm project:

$ git clone -b release_60 https://github.com/llvm-mirror/llvm

Configure LLVM to build as a single static binary:

$ mkdir llvm-build
$ cd llvm-build
$ cmake -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_LINK_LLVM_DYLIB=ON -DBUILD_SHARED_LIBS=OFF ../llvm

Build LLVM:

$ make

Go for a long walk.

If everything goes well, you should find llvm-build/lib/libLLVM.dylib.

@KodrAus thanks. Working on this.

Thanks @mjsabby!

I imagine it wouldn't need to change all that much, but do you think it would be worth setting up a repository for libLLVM that just contains an Azure Pipeline or something for fetching/building an LLVM binary on the supported platforms, bundling those up into a nupkg and publishing to nuget.org? That might give us some more visibility into how the package is produced?

@KodrAus Yup I so desperately want to do that :)

Happy to create a Microsoft sponsored account if you wanted to take a stab at it.

Sure :+1: I'm keen to try spike something out. I'll just spin up a repo and see where it goes. I'll open an issue or send a ping if I get anywhere 😃

So I've actually been able to make some progress on this. Building for OSX and Linux seems to work out ok, but I'm having some trouble getting a single .dll out of the Windows build.

@mjsabby I'm guessing that's what the vcxproj in this repo is all about? Are you able to share some details about how you build LLVM for Windows at the moment?

Yes the vcxproj is for that. In fact, you need to have the code also be built on Linux and macOS. Because that's what the PInvokes call into since neither platform exposes all the arches.

Are you using a hosted Azure Pipeline instance? Let me know the details and we can put all the relevant info in the repo.

Really appreciate you taking this!

Ah right. Sure! The repo I'm working on is here, with the current Azure Pipeline looking something like this. It's still proof-of-concept quality stuff, but does manage to get the native bits produced in some form on Windows, OSX and Linux.

I wasn't able to successfully link up a Windows build using the vcxproj from this repo on my machine, but LLVM takes a really long time to build locally on my poor old Windows box (~3hrs vs ~1hr on the build agent), so I might just try running it in the Windows pipeline after LLVM's build and see how I go getting it to link up there.

@KodrAus do you mind if we copy over your azure pipeline yml over to an MSFT Azure Pipelines? I'll then get beefier VMs for this task.

No problem! Some beefier agents definitely wouldn't hurt. I think the Windows build that's there now is running close to the disk limit.

I've punted on a couple of design questions in the pipeline though, like whether to pull in the LLVM source as a submodule in the repo or just clone it in CI, just to try prove it was even possible to do so once it's all working we might want to revisit them.

Success! The Azure Pipeline I've been working on can now link up a single .dll on Windows. So that's Windows, OSX and Linux now built and bundled in a single pipeline. Whether that .nupkg I'm assembling actually works is a different question, but all the pieces are there now.

@mjsabby Just thinking about next steps here, I'm happy to keep chipping away on my libllvm repo until it gets to a point where we're sure it works in LLVMSharp.

After that, did we want to transfer the libllvm repository somewhere, (is this Microsoft org an appropriate place?), migrate the pipeline to an MS account and set up whatever credentials we'd need to be able to push pre-release packages to MyGet out of CI?

@KodrAus I think we'd want to keep it in the LLVMSharp repo. I was referring to your scripts if they are simple enough to integrate.

We need a cmake project that'll compile the entry points.

@mjsabby Ah gotcha, if you look at the repo they are mostly simple so we should be able to just drop its contents in here somewhere and have it work. There is some organisational complexity in there though that seems like it's worth keeping separate from LLVMSharp itself and could be beneficial for other projects that want raw bindings to LLVM.

So I'd be slightly more in favour of completely shifting the libLLVM build out into its own repository, but I'm happy to move in either direction and could submit a PR that drops them in.

We need a cmake project that'll compile the entry points.

Is this something separate from LLVMs own build project or from the vcxproj we use on Windows for linking up the static libs into a dll?