mymake, C++17, separate compilation, etc

[Quuxplusone]

trafficstars

TravisCI is currently failing with inline variables are a C++17 extension: https://travis-ci.org/github/zenorogue/hyperrogue/jobs/669619228

Could these variables be plain old non-inline variables?

[zenorogue]

The purpose of hooks_build_rvtour being inline is that I want the both of the following to work:

./mymake -std=c++17 rogueviz/staircase which is HyperRogue with rogueviz/staircase added

./mymake -rv which includes all the RogueViz modules

In the second case rogueviz.cpp adds a menu option which invokes a presentation showing all the RogueViz visualizations.

In the first case there is no such presentation, but the staircase visualization can be still enabled from the commandline. Staircase defines a slide for the RogueViz presentation, but since hooks_build_rvtour is defined as inline, "staircase" will still link correctly.

While HyperRogue has to compile with some old compilers, it is not the case with RogueViz, so I would prefer to use C++17 for RogueVIz.

[Quuxplusone]

I noticed when mymake appeared in the repo, but I've never fully understood its purpose. Is it for platforms where make does not exist in the PATH? Is it supposed to run faster than regular make?

If mymake is indispensable: Would it be useful to you if someone were to add those ./mymake lines to the TravisCI build?

Vice versa, would it be useful to you if someone were to add a way to make those targets via Makefile.simple, so that mymake could disappear again?

I've also noticed HyperRogue's slow struggle away from the unity-build model, e.g. introducing the EX macro. I made a few private and abortive attempts to factor stuff out into .h files by hand, but it never seemed worth it because I didn't think such a PR would ever be accepted. Anything you would like to share on your vision/direction in that regard? Are we just waiting for C++20 Modules to be available? ;)

I would prefer to use C++17 for RogueViz

I'll make a note to work on a PR to use -std=c++17 in Makefile.simple when CAP_ROGUEVIZ is enabled.

[Quuxplusone]

Ick, TravisCI's default linux target still by default uses GCC 5.4.0, which has a -std=c++17 mode but does not support inline variables!

https://travis-ci.org/github/Quuxplusone/hyperrogue/jobs/670718104 (GCC 5.4)

However, we can pass dist: bionic to get the current LTS release, and then we get GCC 7.4.0. So I've filed PR #104 with the fix (and other Travis-related fixes).

[zenorogue]

Thanks for the PR! My goals are as follows:

• I want to be able to compile a new version of HyperRogue quickly. Because of the current size of HyperRogue, the unity-build model is much too slow to work on.

• I want to avoid writing header files. They are unnecessary extra work that should be automatized. I have seen Java/C# programmers consider the necessity of writing headers a major disadvantage of C++, and I have also seen that C++20 modules aim to solve this problem, but I could not find any good source on how they intend to do this.

• I want to use the HyperRogue engine to make various visualizations. The hook system makes it possible to modify how HyperRogue works by linking it together with extra files (we use this to make various animations and other experiments). The build system should allow us to do this easily.

• Easily switch between compiling optimized and debug executables.

The program makeh, together with EX/IS/HDR macros, solves the second problem (it is a preprocessor which creates the header file autohdr.h automatically) and I have decided it will be easier to create my own build program (mymake) to achieve the other goals rather than using existing tools. It is not perfect, as it does not detect if struct contents are changed (this requires recompiling everything that used that struct -- it has to be done manually).

The unity-build model is still used by Makefile.simple and other build tools.

[Quuxplusone]

Okay. "Easily switch between compiling optimized and debug executables" seems to be the killer at the moment. Makefile.simple just dumps .o files in the current directory; it could be set up to create an objs/ directory, and even mangle the name of that directory based on the build arguments, like mymake does, but I agree that that might be a worse nightmare to maintain than just using mymake.

I think it would be useful to add an entry to the Travis build matrix showing how mymake is meant to be used. I've got something like that in https://travis-ci.org/github/Quuxplusone/hyperrogue/builds/670845971 — but it does not test the ability to "modify how HyperRogue works by linking it together with extra files."

Re C++20 Modules: It's actually a lot like what you do with EX. You keep your code in separate .cpp files, but you mark all the entities you want to be externally visible with export. (There's also some fiddly but negligible boilerplate; e.g. each .cpp file must start with the word module;.) The big hurdle for HyperRogue will be that C++20 modules are not allowed to contain cyclic dependencies — in C++20 you physically cannot make two modules a and b where a.cpp does import b; and b.cpp does import a;. Because a.cpp can't be compiled until import b; is resolved, which requires compiling b.cpp, which requires resolving import a;, which... Header files can support this kind of mutual dependency; modules cannot.

[KubaO]

I'm not involved in this project at all, but a simple question one could ask is: why not just use cmake and ninja? Ninja is extremely quick. And unity builds are something to speed the CI builds, not to use on developer machines.

[tranzystorekk]

I have played around switching building to cmake here https://github.com/tranzystorek-io/hyperrogue/blob/cmake-rewrite/CMakeLists.txt if anyone wants to take a look etc.