llvm-mingw icon indicating copy to clipboard operation
llvm-mingw copied to clipboard

Collaboration opertunity with Nixpkgs/NixOS

Open Ericson2314 opened this issue 5 years ago • 7 comments

Nixpkgs supports cross compilation to many targets, and I've been trying to add Windows to the list: https://github.com/NixOS/nixpkgs/pull/72366

In the process I've been some patches to LLVM you may be interested in:

  • https://reviews.llvm.org/D69760
  • https://reviews.llvm.org/D69767

Also, rather than use MinGW headers/libs, I've automated the downloading of various Windows SDK / Visual Studio components, through the JSON manifest Microsoft makes available.

Next, I'm trying to get CMake to not get confused with this combination (it was fine until I switched from lld-link to ld.lld, hmm).

Ericson2314 avatar Nov 03 '19 22:11 Ericson2314

I've added WIP mingw support, to try to better match your repo and run into fewer dusty corners. Will keep the visual studio stuff, though.

Ericson2314 avatar Nov 03 '19 23:11 Ericson2314

Hej!

I've automated the downloading of various Windows SDK / Visual Studio components, through the JSON manifest Microsoft makes available.

Oh, that sounds very interesting! I'm following recent releases of MSVC quite a bit, but my process is by downloading and installing them in a windows VM, then zipping up the relevant files and copying the zips to a linux machine, where I run them in wine (with a fairly straightforward setup available at https://github.com/mstorsjo/msvc-wine). If I could avoid the install-on-real-windows-and-zip step it would really make this installation even smoother.

Can you give any pointers to it?

I'll try to comment on your LLVM patches; I tried adding other relevant reviewers when I saw them, but unfortunately I haven't had time to comment on them until now, as I've been travelling.

Regarding comments from the NixOS discussion:

it can do abi-compatable C++ with MSVC

While Clang in MinGW mode doesn't do this (by design),

and has more features implemented such proper C++ threads

libc++ used in MinGW as in this repo does have all that.

Next step might be making a windres-style CLI to llvm-rc, but hopefully that can be put off for a while.

Feel free to use the windres wrapper I have here. It shouldn't require many changes to be used outside of this setup, mainly in how it expects to run the C preprocessor.

mstorsjo avatar Nov 05 '19 13:11 mstorsjo

Can you give any pointers to it?

Sure! The downloading work is in this file: https://github.com/NixOS/nixpkgs/pull/72366/files#diff-f68d2a470af52a513434dc58d7ad943b. In short, from https://aka.ms/vs/16/release/channel there is a link to another json file with URLS and hashes for all the components. (The outer mutable manifest also has a hash for the inner one, but it seems to always be wrong despite the inner one being immutable).

Credit to @est31 for cluing me into this stuff with https://github.com/est31/msvc-wine-rust/.

I'll try to comment on your LLVM patches; I tried adding other relevant reviewers when I saw them, but unfortunately I haven't had time to comment on them until now, as I've been traveling.

No worries, I definitely understand the traveling rush. I need to fix some silly typos in the non-backports too.

While Clang in MinGW mode doesn't do this (by design)

Right. But I still would like to get Clang + LLD a working option in both cases :)

libc++ used in MinGW as in this repo does have all that.

That still uses the windows pthread stuff, right? I did forget about all that, but we already have it packaged so it shouldn't be hard to use.

Ericson2314 avatar Nov 05 '19 15:11 Ericson2314

Can you give any pointers to it?

Sure! The downloading work is in this file: https://github.com/NixOS/nixpkgs/pull/72366/files#diff-f68d2a470af52a513434dc58d7ad943b. In short, from https://aka.ms/vs/16/release/channel there is a link to another json file with URLS and hashes for all the components. (The outer mutable manifest also has a hash for the inner one, but it seems to always be wrong despite the inner one being immutable).

Credit to @est31 for cluing me into this stuff with https://github.com/est31/msvc-wine-rust/.

Thanks! Will look into this later.

libc++ used in MinGW as in this repo does have all that.

That still uses the windows pthread stuff, right? I did forget about all that, but we already have it packaged so it shouldn't be hard to use.

No, it doesn't. When building libc++ for windows, it uses win32 native threading functions directly, both for MinGW and MSVC builds of libc++. My toolchain does bundle winpthreads though, but it's only for convenience as a user requested it.

mstorsjo avatar Nov 05 '19 21:11 mstorsjo

Oh, awesome!

ATM I need to trick down the part of the Windows SDK that has winapifamily.h.

Ericson2314 avatar Nov 06 '19 03:11 Ericson2314

Sure! The downloading work is in this file: https://github.com/NixOS/nixpkgs/pull/72366/files#diff-f68d2a470af52a513434dc58d7ad943b. In short, from https://aka.ms/vs/16/release/channel there is a link to another json file with URLS and hashes for all the components. (The outer mutable manifest also has a hash for the inner one, but it seems to always be wrong despite the inner one being immutable). Credit to @est31 for cluing me into this stuff with https://github.com/est31/msvc-wine-rust/.

Thanks! Will look into this later.

I extended on the same concept, making more of a generic visual studio installer: https://github.com/mstorsjo/msvc-wine/blob/master/vsdownload.py

It tries to be generic (as far as necessary for installing the bits necessary for running the compiler at least), to allow using it for testing out new releases/previews. Thanks for the pointers!

mstorsjo avatar Nov 15 '19 21:11 mstorsjo

Nice!! I will definitely improve mine via it ;)

Ericson2314 avatar Nov 15 '19 22:11 Ericson2314