[Feature] - Add support for cross-platform use

[alan-geller]

Is your feature request related to a problem? Please describe. Currently projects based on Llvm.NET only work on Windows. It would be great if they could be used with .NET Core on Mac and Linux! In particular, I'm using Llvm.NET in a project that is cross-platform, and I can't merge my new feature to master until I have a cross-platform solution.

Describe the solution you'd like The optimal solution would be for there to be Mac and Linux NuGet packages for Llvm.NET, as well as Windows.

Describe alternatives you've considered None

Additional context I'm starting to work on this now. As I understand it from this project's maintainer, it's mostly a build issue. The .NET libraries are all cross-platform by default; the problem is the generated C++ library that wraps the actual LLVM libraries.

[alan-geller]

I'm working in a fork at https://github.com/msr-quarc/Llvm.NET/tree/xplat.

Working on CMake as a first step. I've got a version that seems to work on WIndows; now testing to see if it works on Linux.

[alan-geller]

@smaillet-ms I'm back at this, working on adapting the build scripts to use dotnet instead of msbuild directly. Does the build platform you'd use on Linux and MacOS have PowerShell installed, or should I switch to bash scripts for these platforms?

[alan-geller]

@smaillet Also, any hints on what I need to do in the Llvm.Libs project to make all of this work? Is there anything I need to do other than do a Linux and Mac build and then ask you to add them as assets to the latest Llvm.Libs release?

[1tnguyen]

@alan-geller I've tried the current Llvm.Net NuGet package on Linux and the first thing that threw was the LoadWin32Library call in InitializeLLVM because it will then try to use WinAPI to load the Llvm.Libs DLL. I guess we need to add cross-platform support for this LoadWin32Library as well.

[smaillet]

Sorry for the delayed responses - I'll just say "Life interruptus".

@alan-geller the entire build infrastructure is based on Powershell core, which is viable on Linux/Mac as well. It is generally very isolated from the infrastructure hosting/running the build as that has changed several times over the years. Thus the preference would be to stay with PS core on Linux. Obviously the part that manages the build of the native components will need some work as there is a very specific sequence of things and that is curently all based on MSbuild and not CMAKE. (Documented in the MD files )

@tnguyen-ornl Yes, the load library portion would require re-working to support other OS. However, that code was designed long before dot net core. With .NET 5 due out so soon, now may be the time to drop the legacy desktop and go all up dotnet core/.NET 5 and reset how the interop binding works. Since generating an EXE with .NET core requires selecting the runtime and architecture the packaging system could contain everything in proper folders laid out based on runtime-id. Thus only the DllImports would be needed. [e.g. all the platform specific architecture detection and LoadLibrary cruft can go away] (assuming a naming strategy could be chosen that allows no source changes for the imports...) I've not done that for Linux, but I can't imagine that .NET core itself hasn't already solved the problem of how to do that. [Though as always, that may just be a lack of immagination on my part]

@alan-geller The actual LLVM.libs release gets complicated. Ideally, we'd have a build system available where we could build the libs for consumption by the rest and they wouldn't need publishing. Unfortunately, that requires funding for commercial cloud build accounts that can handle the workload, which this project doesn't currently have. As of now, the libs are built locally by me and published so I know I can trust them. Obviously that doesn't really scale but options are limited. I'm open to suggestions on how we can do this in a reliable, secure and trustworthy manner. (That doesn't require personal out of pocket costs)

[alan-geller]

Current status:

• the CMAKE piece wasn't exporting the proper symbols. I'm researching how to do this cross-platform right now.
• I've replaced the LoadWin32Library using the current .NET Core Native.Load API, which seems to work. It does require .NET Core 3.1 or later or .NET 5, though.
• The PowerShell scripts seem to be good.

@smaillet Any chance you could create the appropriate Llvm.Libs releases for Linux (Ubuntu, if only one) and MacOS?

[swernli]

@alan-geller @smaillet Building off of the work Alan did in the fall, I've got a working cross-platform pipeline up for PR over at #205. This includes updates to the release workflow such that cross-platform build and test can be done there, and the resulting packages pushed to NuGet. See https://github.com/UbiquityDotNET/Llvm.NET/actions/runs/499695323 as an example of the new pipeline execution.

[andyjansson]

Do we have an update on this?

[swernli]

I was able to get a version of cross-platform build to work, but it ended up being too much of a large change for a single PR (see the feedback on #205), as well as making some significant changes to the tooling infrastructure. We started to divide up this large change into smaller chunks for more detailed review (#206 and later #212), but in the meantime for our specific project we ended up finding another solution for cross-platform compatibility. If you have an interest in picking up where I left off, I can provide pointers and branch info on what I did in my attempt, but I think you'd want to coordinate with @smaillet to start from a reasonable place and build on any ongoing infrastructure work he has.