randombit
Clarification: Use of ninja for building
Good day! I have become increasingly frustrated with the long build times for Botan when using vcpkg. I noticed in your documentation that you support the ninja build system now (awesome), but there is a note in there:
https://botan.randombit.net/handbook/building.html#ninja-support
Starting in Botan 3.2, there is additionally support for the ninja build system. Support for ninja is still new and there are probably some rough edges.
Considering we are on Botan 3.10, can ninja be considered "non rough" at this point? VCPKG has "JOM Preferred" set for building but it still seems to be single threaded.
https://github.com/microsoft/vcpkg/blob/92ce24fe9ab55f7e6ad09d337690cd00b6e0f528/ports/botan/portfile.cmake#L108-L134
Lastly. would you be willing to update the vcpkg system or perhaps we can approach this collaboratively. There is currently an open draft PR for Botan 3.10 we could make this update into. https://github.com/microsoft/vcpkg/pull/48188
Re: ninja. I'm using it during development on Linux and macOS, and occasionally on Windows. Some minor things were fixed here and there since Botan 3.2 and, I guess, we could declare it stable by now.
Especially when used for building just the library for packaging I don't expect major problems with ninja anymore.
Re: vcpkg. Personally, I don't have any experience with it. But I could certainly review patches for its Botan recipe. (I'm currently traveling, so response times may be longer, though.)
Re ninja it has also been used for Windows CI builds for some time with no issues. Certainly like anything there might be some bugs remaining, but if there are issues that affected using ninja via vcpkg (or otherwise) certainly we'd work to address them. I think at this point that comment can be removed from the doc.
Re vcpkg as I do not use Windows at all (nor have any background on our CMake integrations) I'm not personally in a position to help with this part.
I think i can manage the conversion, although i just rebuilt botan on windows and Jom did use all my cpu cores. The build still takes well over 20 minutes. I have no idea why it is so slow.
The build still takes well over 20 minutes. I have no idea why it is so slow.
Yeah I can see how that would get old.
Just for comparison purposes on my laptop (i7-1185G7, 4 cores/8 threads, Linux) I cleared the compiler cache and did a clean build, it took GCC 15 slightly over 10 minutes to build library, cli, and tests.
One thing to try would be building via the amalgamation. I think one thing that is probably really hurting on Windows builds is that file access and process execution are both much slower on Windows than on Unix-style kernels. Building as the amalgamation dramatically reduces the number of files read and number of times the compiler is executed. It also nearly eliminates redundant parsing of STL headers/etc which likely is costly.
Another option is, for your usage (AFAIK) you don't need a lot of features we ship - TLS, post-quantum algorithms, X.509 certs, etc. If it's possible to disable these via vcpkg you can skip having to compile them at all which will be a nice savings.
Adding Boost as a dependency adds quite a lot of build time, and only really matters if you're using the asio integration with TLS.
It's possible vcpkg is building, but not installing, the cli and tests. If that's the case maybe they can just be skipped entirely.
If you're not aware of ccache and sccache btw I highly recommend them. No idea if vcpkg supports using a cache, though.
I haven’t tested it yet on this code, but Zig’s compiler can be dropped in as a Clang replacement for most code bases. Two benefits of it: 1) built in caching 2) better and easier cross-compiling (if you’re into that), especially for embedded systems (such as Raspberry Pi).
