mupen64plus-core
mupen64plus-core copied to clipboard
mupen64plus
[WIP] Preliminary support of meson builds.
In order to lower the maintenance burden of the build system (eg. keeping in sync both Makefile and Visual studio files) I've explored the possibility of migrating to the meson build system [1]. So, I've ported the original Makefile to meson with some minor arrangements to fit in the new paradigm. What I'm looking for now is feedback from other developers to see if there is interest in that work and what can be done to ease the eventual transition to it. I've tested this on my machine (linux, x64) and I'd like to know if it works in other platforms (windows, bsd, mac), cpu (arm, x86, mips?, ppc?) and dev env (cross-compilation, msvc).
In order to test (assuming the use of the ninja backend):
cd <path_to_m64p-core>
mkdir build
cd build
meson ../
ninja
You can also inspect and change build options with
meson configure
Note that this build system relies on python3 which might not be available by default in windows (should we add it to win32-deps repo ?).
Please report any issue, feedback or comment. When/If we get this repo into shape with meson, I'll also port other repo to it (should be more easy).
cc: @richard42
[1] http://mesonbuild.com/
Tried on windows with msvc but it fails, here's the log:
The Meson build system
Version: 0.43.0
Source dir: D:\Projects\GSI\mupen64plus\mupen64plus-core
Build dir: D:\Projects\GSI\mupen64plus\mupen64plus-core\build
Build type: native build
Project name: mupen64plus-core
Native C compiler: cl (msvc 18.00.31101)
Native C++ compiler: cl (msvc 18.00.31101)
Build machine cpu family: x86
Build machine cpu: x86
Compiler for C supports argument -ffast-math: NO
Compiler for C supports argument -fno-strict-aliasing: NO
Compiler for C supports argument -fvisibility=hidden: NO
Compiler for C supports argument -Wno-unused-function: NO
Compiler for C supports argument -Wno-unused-parameter: NO
Compiler for C++ supports argument -ffast-math: NO
Compiler for C++ supports argument -fno-strict-aliasing: NO
Compiler for C++ supports argument -fvisibility=hidden: NO
Compiler for C++ supports argument -Wno-unused-function: NO
Compiler for C++ supports argument -Wno-unused-parameter: NO
Compiler for C++ supports argument -fvisibility-inlines-hidden: NO
Found Pkg-config: NO
Meson encountered an error in file meson.build, line 192, column 0:
Pkg-config not found.
Also I'm not sure to understand, is this also going to generate a visual studio project file?
@Gillou68310 Thanks for giving a try :)
By default, meson uses the ninja backend, but you can also tell it to generate a vs project file using the --backend {vs,vs2010,vs2015,vs2017} You might have to run this under the vs command prompt [1].
As for your error about missing pkg-config, I'll need to look a bit deeper into it. Indeed, meson rely on pkg-config to find dependencies. This works well in linux, but in windows, no so much. Two solution here: either we go the source route and use wrap files, or we go the prebuild binary way [2].
[1] http://mesonbuild.com/Using-with-Visual-Studio.html [2] http://mesonbuild.com/Shipping-prebuilt-binaries-as-wraps.html
My initial thought on this is that I don't like it. Makefiles and MSVC project files are "native" on linux/bsd/windows, so generally they're easiest to use for end users. Adding another system on top adds new things to break and additional dependencies, plus a new thing for developers to learn and maintain.
I think having separate project/make files for linux and windows is not as much of a hassle for developers as the fact that we have so many different modules, each with their own build system. So if a developer is working on a single module, like for instance RSP-HLE, and he or she adds a new file, this only requires changes in 2 places. But if a developer makes a general build system change (like, adding a new unix-like target system to the makefile) it requires changes to N files in N repositories, which is a bigger pain. Going to meson isn't necessarily going to address this issue.
Some projects do have success with other system like cmake, so I'm not saying that I would never accept this type of change. But it's going to be difficult, because at a minimum we need it to support all of the systems on which we currently build, and not introduce any additional complexity.
@richard42 Thanks for joining the discussion
meson can generate ninja files (equivalent to Makefiles), VS solutions (2010, 2015 and 2017) files and XCode files. So normally each developer should get something native on their plateform. Then adding/removing/moving a file is just a matter of editing the meson.build file (eg just as easy as what it is with our current Makefile, and much easier than editing manually a vs project+filter file).
Learning meson is not that hard. I've made this work in less than a week, working on it while commuting. The documentation is okay and in case of doubt there are many unit tests to learn from.
The point you raise about having many repo for each plugin is valid and I think here meson can help a little bit : if we have only 1 meson.build to edit per repo, that's half what we have to edit now (1 Makefile, 1vsproj). Also because meson.build files are "composable" (you can embed projects inside another) we might be able to switch to a single top repo with all core+ui+plugins as subprojects and factor some common build logic in the top project. This could maybe address the current maintenance issue.
Also some benefits I see with this transition :
- many projects have reported a much better build time by switch to meson, especially on Windows.
- better tracking of the compilation chain, meaning that when I make a change in a file or in a build option, meson will only rebuild what's really needed. I don't need to do a "make clean & make DEBUG=1 all" all the time. This speeds up my dev flow quite a bit.
- dependency management looks good also. With the possibility to either use the system dependencies or some fallback (be it from source or some prebuilt objects).
- Migration to other newer Visual Studio version will not require a significant effort anymore, just switch backend. And when I add a new directory, I don't need to care about some UniqueIdentifier in the filter file anymore.
Anyway, we don't have to switch to meson overnight. It can coexists (eventually in a separate branch if you don't wan't it in master yet) for some time until it has proven its value to other devs.
There are some interesting potential advantages there. However I would suggest moving the meson files into a sibling directory of the other projects, ie /project/meson/, instead of the root, to avoid cluttering up the base folder.
Rebased against master. Tried to use the fallback parameter for dependencies. Tried to fix some concerns from @bentley. @richard42 I'm not sure meson supports meson.build in non source root directory.
Moved some "external" files outside of src dir, but didn't update Makefile and VisualStudio. So build failures with them is expected.
Rebased against master, addressed some concerns from @bentley, and tried to update CI files, but it looks like meson documentation is not up-to-date (http://mesonbuild.com/Continuous-Integration.html). Will look into that later. @Gillou68310 Can you try again building with msvc to see if the fallback subprojects work please ?
@Narann I've done an epic fail while trying to get appveyor to build (starting an interactive python env...) and I'd like to cancel the build, but I can't since I don't have the rights, can you add me the "Run Project builds" role so that I can cancel this jobs, otherwise our Appveyor instance will be unavailable for several hours (each jobs has a 1 hour time quota). Thanks
Hi @bsmiles32, sorry for delay, I've added the whole mupen64plus/Maintainers group as admin of appveyor, this mean you and @Gillou68310 have access to the project. :)
Tell me if you find what you need with this!
I managed to generate the msvc solution with osd=false.
Looks awful:
This will hardly be usable for development purpose IMO.
Sorry if I sound rude but this reminds me cmake and god I hate cmake :-)
Anyway here's the build error log:
Error 239 error MSB3073: The command ""C:\Program Files (x86)\Meson\meson.exe" "--internal" "exe" "C:\Projects\GSI\mupen64plus\mupen64plus-core\build\meson-private\meson_exe_nasm.exe_047df892b97caaff0a74baa7b4fce314c91bbf73.dat"
:VCEnd" exited with code 1. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.CppCommon.targets 202
Error 529 error LNK1104: cannot open file '..\..\subprojects\zlib-1.2.8\z.lib' C:\Projects\GSI\mupen64plus\mupen64plus-core\build\subprojects\libpng-1.6.17\LINK
Error 555 error C1083: Cannot open include file: 'zlib.h': No such file or directory C:\Projects\GSI\mupen64plus\mupen64plus-core\subprojects\libpng-1.6.17\pngtest.c 72
Error 1638 error C2143: syntax error : missing ')' before '(' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1639 error C2091: function returns function c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1640 error C2143: syntax error : missing ')' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1641 error C2143: syntax error : missing '{' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1642 error C2059: syntax error : 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1643 error C2059: syntax error : ')' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 43
Error 1644 error C2143: syntax error : missing ')' before '(' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1645 error C2091: function returns function c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1646 error C2143: syntax error : missing ')' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1647 error C2143: syntax error : missing '{' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1648 error C2059: syntax error : 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1649 error C2059: syntax error : ')' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 52
Error 1650 error C2143: syntax error : missing ')' before '(' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1651 error C2091: function returns function c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1652 error C2143: syntax error : missing ')' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1653 error C2143: syntax error : missing '{' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1654 error C2059: syntax error : 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1655 error C2059: syntax error : ')' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 62
Error 1656 error C2143: syntax error : missing ')' before '(' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1657 error C2091: function returns function c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1658 error C2143: syntax error : missing ')' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1659 error C2143: syntax error : missing '{' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1660 error C2059: syntax error : 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1661 error C2059: syntax error : ')' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 72
Error 1662 error C2143: syntax error : missing ')' before '(' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1663 error C2091: function returns function c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1664 error C2143: syntax error : missing ')' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1665 error C2143: syntax error : missing '{' before 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1666 error C2059: syntax error : 'string' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1667 error C2059: syntax error : ')' c:\projects\gsi\mupen64plus\mupen64plus-core\src\api\m64p_common.h 82
Error 1668 error C2143: syntax error : missing ')' before '(' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1669 error C2091: function returns function C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1670 error C2143: syntax error : missing ')' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1671 error C2143: syntax error : missing '{' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1672 error C2059: syntax error : 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1673 error C2059: syntax error : ')' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 34
Error 1674 error C2143: syntax error : missing ')' before '(' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1675 error C2091: function returns function C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1676 error C2143: syntax error : missing ')' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1677 error C2143: syntax error : missing '{' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1678 error C2059: syntax error : 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1679 error C2059: syntax error : ')' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 66
Error 1680 error C2143: syntax error : missing ')' before '(' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
Error 1681 error C2091: function returns function C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
Error 1682 error C2143: syntax error : missing ')' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
Error 1683 error C2143: syntax error : missing '{' before 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
Error 1684 error C2059: syntax error : 'string' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
Error 1685 error C2059: syntax error : ')' C:\Projects\GSI\mupen64plus\mupen64plus-core\src\api\common.c 101
@Narann Thanks for looking into it. I haven't figured out how to cancel a job, but I got added to the administrator team, so I don't know...
@Gillou68310 Thanks for taking the time to try it. I'll try to make it work in appveyor and see how it goes from here. I have yet to explore all the cross compilation aspect of it. Sadly as you said, it indeed looks like a cmake generated solution... not sure what can I do about it. Yet if it avoids all the pain of either clicking all the vs menus or editing the always changing sln/vcproj files...
Mhhh, you should have admin rights if you sign in with github account. Do you see the cancel build button once signed in?
https://ci.appveyor.com/project/Narann/mupen64plus-core/build/1.0.291
Looks like you where in a sub job, if you would have clicked in "Latest build" you would be in the main job and should be able to manage the whole test.
For example, now you should be able to rerun the build:
Click on your name, Account details, what do you see in "GitHub authorization"? I have this:
AppVeyor was authorized 3 months ago for ... access to GitHub repositories as Narann account. Your account authorizations.
I have this
AppVeyor was authorized 2 hours ago for admin:repo_hook,read:org,repo:status access to GitHub repositories as bsmiles32 account. Your account authorizations.
I think you should look inside your "Team" menu item and work from there to allow me to run jobs
Looks like there was an "organization" webhook to set in the mupen64plus organization to make things work.
Updated against master, but still struggling with CI (both travis and Appveyor)... Will retry at a later date.
Yeah ! First time TravisCI and AppVeyor build simultaneously :)
Still a bit WIP as I still have to disable various dependencies when building in VS2013 (eg OSD OpenCV) under AppVeyor. I also need to test building under mingw32. Also there are quite a few warnings in the logs...
x86 versions (new_dynarec and hacktarux dynarec) also builds on Windows ! Had a hard time finding why it was failing before.... just to find out that it was because of a bug in nasm (-MD option prevented other options such as -D LEADING_UNDERSCORE to be taken into account)...
I guess to reach "parity" with current Makefile+VS2013 I need to have the OSD build in Windows, and have MINGW32 builds in TravisCI.
I think I've now reached parity with Makefile+VS2013 projects as both now build on Appveyor and TravisCI. I also got the MXE cross compilation working on Travis. However, I had to disable LTO with Clang and MXE. For the former I hit LLVMgold plugin error, which I think is due to using an old version of Ubuntu. For the latter, I get unresolved symbols on my own subprojects symbols (md5, minizip, oglft, ...) and some warning about LTO and "ar". So I disabled these for now. Freetype library has not been yet properly imported as a meson subproject, so you either need to get it through pkg-config or use the prebuilt version in mupen64plus-win32-deps. As a small bonus I added the generation of pkg-config file to ease consumption of this library by others. This has been asked recently https://github.com/mupen64plus/mupen64plus-core/issues/562.
I could still do some more cleanings and refinements here and there, but I feel like asking for some feedback first could be beneficial.
My porting to meson has been quite good overall even if a bit hard at times :
- the meson documentation and test suite are very good resources to learn how to do things with meson
- it has helped me finding some errors and bugs such as #583, #581 and we got a huge list of warnings now (maybe I should reduce the warning level for now to avoid compiler spam for casual user/contributor, and tackle in my own fork the warnings fixing)
- the first difficulty was getting dependencies right. meson subprojects and wraps have been really helpful here and I'd like to go a little bit further so we can have all of our dependencies as subprojects (and even some as wrap). These would allow full flexibility regarding dependencies, eg. have them static or dynamic, have them properly configured (which features to enable/disable) ... without relying on the package manager which some system don't support (Windows, iOS?).
- the second difficulty was getting things to build on Appveyor and TravisCI. Apart from the difficulty of getting the dependencies right, the main issue was circumventing the fact that these mostly provide old version of software (ninja, meson). So I need to fetch them from elsewhere which makes the build and configuration more complex (and longer) than it should be. Yet we could mitigate these downloads times by caching some of them, I think Appveyor and TravisCI support that.
So yeah, now is a good time to test this PR, so I can fix the remaining concerns. cc : @richard42, @loganmc10 @Gillou68310 @bentley @fzurita
I've also had some success building using meson on my Raspberry Pi 3 (aarch64). There are still some stuff that are not supported (no_asm=1) and some dependencies still have some troubles getting compiled as subprojects from meson (~~libpng~~ and sdl2) but eventually will get there :)
update: PR pending for libpng (https://github.com/mesonbuild/libpng/pull/9)
I'm messing with a little side project that consumes mupen64plus and just want to register my support for this change, though I know it's an old one! Being able to reference this as a meson subproject would greatly simplify my build.
Were there any significant blockers to getting this merged, or does it just need some help across the finish line?
it's not just a little piece of work to get it across the finish line; it's more like we need a champion to take this on and commit to maintaining it in the future, because I'm happy enough just continuing with the MSVC and makefile build systems. I don't have interest in learning and maintaining an entirely different build system at this point.