Compile and test natively on Windows

[sfiera]

Some parts of Antares are already compiled for Windows (anything that isn’t explicitly excluded in https://github.com/arescentral/antares/blob/master/BUILD.gn). However, this is done by cross-compilation from Windows, and the tests are run in WINE.

In an ideal world, it would be possible to compile and run Antares (or at least parts of it) natively on Windows.

Parent issue: #79

[elasota]

I can probably take a look at this some time soon-ish.

[sfiera]

Thanks. I wouldn't really have an idea where to begin. I'm happy to field any questions; I suspect the first hurdle will be getting gn and ninja binaries (from Chromium, I suppose).

[elasota]

Well, first question I'd have is if the out/cur symlink is necessary or if that can be done some other way. Symlinks in Windows require administrator which I'd prefer to avoid if possible and I can't tell if it actually needs the symlink after configure.

Working on it slowly, having to tweak some things to get the libs compiling in VS. Biggest issue right now is pn::output::format will not compile because VS doesn't like casting a list to char array, but constructing it in a temp causes an internal compiler error. (https://developercommunity.visualstudio.com/t/Internal-compiler-error-when-using-param/1654868)

[sfiera]

It’s not necessary. It’s just a way to remember which build directory is currently in use. I believe the only places that rely on it are:

• Makefile
• scripts/test.py (could be replaced by a flag from Makefile)
• scripts/notarize (Mac only)
• scripts/gen-install.py (Linux only)

Is it not possible to compile with clang on Windows as well?

[elasota]

Is it not possible to compile with clang on Windows as well?

It may be, but I think it's best to do this in a way that's as "normal" as possible.

Plus some of the errors that it's complaining about (calling non-constexpr functions from constexpr functions, missing return values) are valid.

Unfortunately it looks like the ICE in Procyon is due to the "write_args(arg)..." expansion which is really bad since there's basically no workaround except for manually specializing (which kind of defeats the entire purpose), so gonna have to wait for Microsoft to fix it.

[assertivist]

Hey @elasota, I work on a similar porting project (game developed for Macs pre-OSX ported to modern systems mostly on Linux). I have found that MSVC is an extremely cranky compiler. You appear to be running into this already! I just wanted to toss out some options that you might try other than fighting with MSVC.

We had the best luck taking our Makefile (which is "Linux and Mac first") and porting it into MSYS2, then building with clang/mingw. This is how our current windows build gets created. It might be an option to look into. It provides a mostly POSIX-compliant build environment on Windows, and they have a ninja package. I personally was able to get the farthest building Antares here. I know I was able to get the base dependencies to build. I don't know how "normal" this is, compared to using VS.

The other thing we've tried is using cmake within Visual Studio. This support has actually improved quite a bit over the past few years. We haven't seen any real difference between the results of either process, so we have been sticking to MSYS2 as it is way easier to automate for CI/CD.

Thanks for taking a look at it in any case.

[elasota]

I'd rather just get the POSIX dependencies out, which I'm almost done with. Ironically the biggest incompatibility is that ssize_t isn't available (so I'm just going to use a typedef that uses ptrdiff_t in VS instead).

cmake would be nice just because it's easier to use within VS. GN's VS support appears to be broken and not actively supported.

I almost have it compiling (just need to fill out win-dirs.cpp I think), and made a workaround for the ICE in Procyon. If I get it working then I'll make some PRs.

It looks like some of the tests need parameters to work, how do you normally run them?

[sfiera]

My main goal is to avoid breakages when making changes across platforms. A simple way to do that would be to use the same compiler, but as long as the CI tests can catch problems it’s not a requirement.

The tests are run from scripts/test.py, which has four categories of tests:

• unit tests (color-test, editable-text-test, and fixed-test), which run without arguments,
• replay tests (the largest category), which run .../replay test/$NAME.NLRP --text --output=$OUT and diff text renderings against test/$NAME,
• offscreen tests, which simulate user input in the UI and diff text or image renderings against test/$NAME, and
• data tests, which produce some sort of output and diff that against test/$NAME.

The last two categories are not very homogeneous. Many of the tests have a “smoke test” version which omits the output to run faster (possibly less important than it was 10 years ago).

[elasota]

Running into another problem, not sure of the best solution. A lot of the GN configs set cflags+ldflags directly, which is bad for VS because many gcc flags will cause CL and LINK to error out. I've been working around this in my build by adding a "compiler_type" var in the BUILDCONFIG.gn, but pretty much everything that sets compiler flags needs to account for it.

So, I can integrate the required changes (and the VS boilerplate) to gn-tools if you want, but then all of the other BUILDCONFIG.gn's will have to be updated to declare compiler_type.

[sfiera]

I’m not quite following. Are you suggesting that the definition of e.g. :antares_private be something like:

config("antares_private") {
  include_dirs = [
    "include",
    "$target_gen_dir/include",
  ]
  if (compiler_type == "msvc") {
    cflags = […]
  } else {
    cflags = ["-Wall", …]
  }
}

Is it possible to use current_toolchain == "/build/lib/msvc"?

[sfiera]

Travis is now re-enabled with OSS build quota, though I’m wondering if it might be better to drop it and use GitHub actions.

[elasota]

In most cases it looks like: if (cflags_type == "gcc") { cflags = [ "-Wall", "-Werror", ] } else { cflags = [] } ... or similar. Been checking some options and I think an import is probably the best bet, and friendlier to cross-compile configs.

I'll PR it tomorrow once I've cleaned it up a bit.

[sfiera]

I don’t think an else should be needed for empty cflags. And is it possible to use current_toolchain instead of introducing a new variable?

[elasota]

OK I'll do that instead.

I was able to get it running

Need to get through all of the submodule updates though. Almost everything's requiring some tweaks.

[sfiera]

Congratulations! That's great.

[sfiera]

CI is now migrated over to GitHub Actions, and includes Windows builds (in antares, procyon, and libsfz). Not everything is enabled on the Windows builds yet, and I’m sure that there are better ways to support Windows than what I did.