YimMenuV2 icon indicating copy to clipboard operation
YimMenuV2 copied to clipboard

Working cross-compilation

Open playday3008 opened this issue 6 months ago • 16 comments

Tested on Fedora 41, and Ubuntu 24.04.

Utilizes either Zig or Clang with MinGW.

  • Using Zig works nice, like 'Plug&Play`.
  • Using Clang with MinGW, works kinda fine.
    • Depending on distro, can be slightly borked, for example Ubuntu 24.04 provides d3d12.h which misses typedef required for ImGui compilation. (that typedef should be there), PR to ImGui is already open, ocornut/imgui/#8702 (already closed).

Though I'm thinking to just drop Clang support from cross-compilation, as it requires lld and MinGW, which are separate packages, while standalone Zig can build it without any problems no matter where, Windows/Linux/macOS.

playday3008 avatar Jun 19 '25 04:06 playday3008

Using Zig works nice, like 'Plug&Play`.

What does Zig do here exactly? I don't think it contains a C++ compiler

maybegreat48 avatar Jun 19 '25 04:06 maybegreat48

It does image How else workflow would succeed under link in first comment

playday3008 avatar Jun 19 '25 04:06 playday3008

OK so according to their website Zig just uses LLVM under the hood to compile foreign code. Why can't we just use Clang then, since it's the official LLVM frontend for C++? I don't really care that much but I find it funny that we're using Zig to compile a C++ codebase

maybegreat48 avatar Jun 19 '25 04:06 maybegreat48

To compile project under Linux, Clang requires MinGW, which isn't under LLVM or Clang, it's independent, each distro can put anything into MinGW package (and name it in anyway), and Clang (CMake kinda too) needs to figure out how to work with that, see example with ImGui and d3d12.h, worked fine on my Fedora 41, while under Ubuntu 24.04 it failed due to missed stuff there

while

To compile project under Windows/Linux/macOS, Zig requires Zig, that's all.

To be honest, that whole cross-compilation stuff can be omitted if Zig were used as primary compiler.

playday3008 avatar Jun 19 '25 04:06 playday3008

To compile project under Linux, Clang requires MinGW, which isn't under LLVM or Clang, it's independent, each distro can put anything into MinGW package, and Clang needs to figure out how to work with that, see example with ImGui and d3d12.h, worked fine on my Fedora 41, while under Ubuntu 24.04 it failed due to missed stuff in MinGW

Makes sense. I'll wait for @Yimura's review since this changes the default CI workflows. Compiling the project this way doesn't add a garbage collector or anything unexpected, right?

To be honest, that whole cross-compilation stuff can be omitted if Zig were used as primary compiler.

I guess, but we have to use their build system which means you have to introduce Zig into our codebase, which might cause unforeseen issues and break a lot of the tooling relying on the cmake files

maybegreat48 avatar Jun 19 '25 05:06 maybegreat48

Makes sense. I'll wait for @Yimura's review since this changes the default CI workflows. Compiling the project this way doesn't add a garbage collector or anything unexpected, right?

Nope, that's like Clang, but just works

playday3008 avatar Jun 19 '25 05:06 playday3008

I guess, but we have to use their build system which means you have to introduce Zig into our codebase, which might cause unforeseen issues and break a lot of the tooling relying on the cmake files

Kind of, it's sure a dependency, but the same way the Clang/MSVC is a dependency, and I'm not sure how could it cause any CMake related issues, when used as zig cc it behaves the same way as Clang.

But if such move can be done, it for sure needs to be tested, at least on Windows, because code I wrote is for sure Linux compatible, but maybe it wouldn't be hard to adapt it to Windows as well

playday3008 avatar Jun 19 '25 05:06 playday3008

Kind of, it's sure a dependency, but the same way the Clang/MSVC is a dependency, and I'm not sure how could it cause any CMake related issues, when used as zig cc it behaves the same way as Clang

Can you get CMake to set up Zig instead of making users install it manually?

But if such move can be done, it for sure needs to be tested, at least on Windows, because code I wrote is for sure Linux compatible, but maybe it wouldn't be hard to adapt it to Windows as well

I use Clang (LLVM) to compile the menu and it works well, so I assume the Zig compiler would too

maybegreat48 avatar Jun 19 '25 05:06 maybegreat48

Can you get CMake to set up Zig instead of making users install it manually?

That's possible, could you test it when I'm done? And maybe use more user friendly conversation app, Dicsord maybe?

I use Clang (LLVM) to compile the menu and it works well, so I assume the Zig compiler would too

No, I mean test tooling itself, not compiled binary, but process of compilation itself

playday3008 avatar Jun 19 '25 05:06 playday3008

The reason compiling with mingw/clang with windows-gnu target isn't supported is because the menu will be ABI incompatible with the game. Almost all virtual function calls will break and crash the game. See: https://github.com/YimMenu/YimMenu/pull/562

To compile with clang on linux we must set the target to windows-msvc and use msvc headers from msvc-wine.

tupoy-ya avatar Jun 19 '25 05:06 tupoy-ya

某种程度上,它肯定是一个依赖项,但就像 Clang/MSVC 是一个依赖项一样,我不确定它如何导致任何与 CMake 相关的问题,当启动 zig cc 时,其行为与 Clang 相同

您允许 CMake 设置 Zig 而不是让用户手动安装它吗?

但如果这样的举动能够实现,肯定需要进行测试,至少在 Windows 上,因为我写的代码肯定与 Linux 兼容,但也许将其装备到 Windows 上也难

我使用 Clang (LLVM) 来编译菜单,效果很好,所以我假设 Zig 编译器也能

请问yim为什么进公开战局掉单?因为开BE了

hjkl123456789 avatar Jun 19 '25 05:06 hjkl123456789

Zig supports windows-msvc but only without libc, I'll look into it

playday3008 avatar Jun 19 '25 05:06 playday3008

Is compilation the only thing that has been tested? Or has a Zig compiled binary been tested against GTA V?

As tupoy-ya mentioned, we went with Clang because that's what the game is currently built with. We also need to think about the additional technical debt this brings, on one hand the compilation stack is simplified on the other hand we've got a whole lot more CMake code to maintain in case something breaks in the future.

Yimura avatar Jun 19 '25 21:06 Yimura

Is compilation the only thing that has been tested? Or has a Zig compiled binary been tested against GTA V?

tldr: Zig compiled binary works with GTA V Tested with GTAV, didn't crash, injected at menu, loaded into game, wondered in menu and game, works fine.

Check for yourself: https://github.com/playday3008/YimMenuV2/actions/runs/15769154573

As tupoy-ya mentioned, we went with Clang because that's what the game is currently built with. We also need to think about the additional technical debt this brings, on one hand the compilation stack is simplified on the other hand we've got a whole lot more CMake code to maintain in case something breaks in the future.

Yeah, that's true, although, Clang is frontend for LLVM, as well as Zig, so, backend is the same. tooling is just better. Also, I'm seeing Zig compiler as Docker from the world of compilers, it just works, no problem with some strange Linux distros, no problem with MSVC versions, etc. Also CMake is kind of a mess of itself, but thats not about CMake.

Should be noted, I tried to make Zig build x86_64-windows-msvc and it's like 100x complicated, because as soon as CMake sees that it supports _MSC_VER definition it tries to talk to zig cc/zig c++ as to MSVC compatible compiler, and it just fucks it up, I made it compatible, but I wish I didn't. But that's talk for another time.

EDIT: Played for couple of hours with Zig compiled version, works just fine

playday3008 avatar Jun 19 '25 23:06 playday3008

@Yimura Rerun workflow, now all of them should succeed. Couldn't test if that was working because self-hosted runner is used.

playday3008 avatar Jun 20 '25 16:06 playday3008

Download the artifacts for this pull request:

Which option is better for end-user?

CanerKaraca23 avatar Jun 21 '25 09:06 CanerKaraca23

Which option is better for end-user?

In theory, Native, as it guarantees MSVC ABI compatibility, on practice, all of them works the same (conclusion from my testing)

It should affect only developers, by allowing equally painless DX.

playday3008 avatar Jun 21 '25 09:06 playday3008

Clang cross-compile failing.

[241/309] Building CXX object CMakeFiles/YimMenuV2.dir/src/game/frontend/submenus/Recovery/StatEditor.cpp.obj
FAILED: [code=1] CMakeFiles/YimMenuV2.dir/src/game/frontend/submenus/Recovery/StatEditor.cpp.obj 
/usr/bin/clang++ --target=x86_64-w64-mingw32ucrt -DCXX_FORMAT_SUPPORT -DIMGUI_DEFINE_MATH_OPERATORS -DNOMINMAX -DWIN32_LEAN_AND_MEAN -DYimMenuV2_EXPORTS -D_CRT_SECURE_NO_WARNINGS -I/home/runner/work/YimMenuV2/YimMenuV2/src -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/imgui-src -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/minhook-src/include -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/minhook-src/src/hde -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/asynclogger-src/src -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/asynclogger-src/include -I/home/runner/work/YimMenuV2/YimMenuV2/build/_deps/json-src/single_include -O2 -g -DNDEBUG -std=gnu++23 -fms-extensions -femulated-tls -fuse-ld=lld -fno-rtti -Wno-date-time -Winvalid-pch -Xclang -include-pch -Xclang /home/runner/work/YimMenuV2/YimMenuV2/build/CMakeFiles/YimMenuV2.dir/cmake_pch.hxx.pch -Xclang -include -Xclang /home/runner/work/YimMenuV2/YimMenuV2/build/CMakeFiles/YimMenuV2.dir/cmake_pch.hxx -MD -MT CMakeFiles/YimMenuV2.dir/src/game/frontend/submenus/Recovery/StatEditor.cpp.obj -MF CMakeFiles/YimMenuV2.dir/src/game/frontend/submenus/Recovery/StatEditor.cpp.obj.d -o CMakeFiles/YimMenuV2.dir/src/game/frontend/submenus/Recovery/StatEditor.cpp.obj -c /home/runner/work/YimMenuV2/YimMenuV2/src/game/frontend/submenus/Recovery/StatEditor.cpp
/home/runner/work/YimMenuV2/YimMenuV2/src/game/frontend/submenus/Recovery/StatEditor.cpp:426:128: error: no member named 'to' in namespace 'std::ranges'
  426 |                                                 auto components = TrimString(std::string_view{line.begin(), line.end()}) | std::ranges::views::split('=') | std::ranges::to<std::vector<std::string>>();
      |                                                                                                                                                             ~~~~~~~~~~~~~^
/home/runner/work/YimMenuV2/YimMenuV2/src/game/frontend/submenus/Recovery/StatEditor.cpp:426:155: error: expected '(' for function-style cast or type construction
  426 |                                                 auto components = TrimString(std::string_view{line.begin(), line.end()}) | std::ranges::views::split('=') | std::ranges::to<std::vector<std::string>>();
      |                                                                                                                                                                             ~~~~~~~~~~~~~~~~~~~~~~~~^
/home/runner/work/YimMenuV2/YimMenuV2/src/game/frontend/submenus/Recovery/StatEditor.cpp:426:157: error: expected expression
  426 |                                                 auto components = TrimString(std::string_view{line.begin(), line.end()}) | std::ranges::views::split('=') | std::ranges::to<std::vector<std::string>>();
      |                                                                                                                                                                                                       ^
3 errors generated.

eXhumer avatar Jun 24 '25 20:06 eXhumer

@eXhumer, your comment should've been made an issue.

Yimura avatar Jun 24 '25 20:06 Yimura

Clang cross-compile failing.

I'm using experimental C++26 features. There should be a flag to enable that on Linux Clang

maybegreat48 avatar Jun 24 '25 20:06 maybegreat48

Clang cross-compile failing.

I'm using experimental C++26 features. There should be a flag to enable that on Linux Clang

Is it? https://en.cppreference.com/w/cpp/ranges/to.html seems to imply this being a C++23 feature.

eXhumer avatar Jun 24 '25 20:06 eXhumer

Is it? https://en.cppreference.com/w/cpp/ranges/to.html seems to imply this being a C++23 feature.

True, they added it in C++23 but I think you need parts of C++26 for this usecase. Anyway, considering the Zig build works, there should be a way to get Clang to work too

maybegreat48 avatar Jun 24 '25 20:06 maybegreat48