cpp-httplib icon indicating copy to clipboard operation
cpp-httplib copied to clipboard

Add C++ modules support

Open mikomikotaishi opened this issue 4 weeks ago • 14 comments

This pull request adds support for C++20 modules through CMake. It is enabled by the HTTPLIB_BUILD_MODULES option (which requires HTTPLIB_COMPILE to be enabled, though it probably doesn't have to - I only forced this requirement because it seems to make the most sense to force the library to compile if modules are to be compiled).

mikomikotaishi avatar Dec 08 '25 04:12 mikomikotaishi

Great work. I also thought about this.

The approach is very similiar to what was done to https://github.com/nlohmann/json with https://github.com/nlohmann/json/pull/4799. Then, I noticed ...

Is this temporary approach with a file with using ... for the meantime while modules and no modules are supported?

This approach is not using modules natively but rather as an interface to the original way.

Does this method work without disadvantages?

Spixmaster avatar Dec 08 '25 09:12 Spixmaster

This is the best way to my knowledge to support modules on top of a header-only or header/source library, allowing continued support for older versions while providing newer features as an option.

I'm not aware of any disadvantages to it besides a being additional translation unit to compile, but if I am wrong please correct me. The only glaring difference in API is that detail symbols are hidden as they are not exported, but in my opinion that's probably better not to expose detail symbols and flood IDE suggestions with implementation details.

mikomikotaishi avatar Dec 08 '25 09:12 mikomikotaishi

What about compiled libraries? Is it possible to have the traditional method and modules installed in parallel?

I am thinking of repositories that ship compiled .so or .a.

This is relevant here, https://aur.archlinux.org/packages/cpp-httplib-compiled .

Spixmaster avatar Dec 08 '25 10:12 Spixmaster

Yes I believe it's possible to use shared/static libraries with modules, all of my modular projects compiled to shared libraries that an executable consumes

mikomikotaishi avatar Dec 08 '25 10:12 mikomikotaishi

@mikomikotaishi, thanks for the fine pull request! It's fantastic, but my concern is that someone needs to update modules/httplib.cppm whenever new external symbols are added or existing symbols are removed/renamed, and it could happen quite often. I am not planning to maintain this file. So if you cannot keep maintaining the file, I am probably not merge it since the file can be easily out-of-date. Is there a way to generate modules/httplib.cppm from httplib.h automatically?

@sum01 @jimmy-park @Tachi107 do you have any thought about this pull request?

yhirose avatar Dec 08 '25 23:12 yhirose

I could create a Python script, or some other kinds of automated means of updating, which you could run every time it is updated. Until then I would be OK with maintaining this file, as it is a simple process. Such a script would probably comb through the file and add any symbols not part of a detail or internal namespace, or prefixed with an underscore, etc.

However, I am curious why it is not feasible to update the file manually. In case it isn't clear how, one can update the file by adding a using httplib::NEW_SYMBOL_HERE; into the export namespace httlib { } block, for each new symbol that is added. Do let me know if any more information is needed.

mikomikotaishi avatar Dec 08 '25 23:12 mikomikotaishi

I have also seen some repositories use bots to push some commits too. Potentially one such bot could be set up to automatically populate the module with new changes each time there is a mismatch. I don't know anything about how to set this up, but I have seen this before and it could potentially be a solution (but I think the simplest one is just to run a Python script each time any update to the library happens).

mikomikotaishi avatar Dec 08 '25 23:12 mikomikotaishi

Anyway, I think this could be one such way of automatically updating the module.

mikomikotaishi avatar Dec 08 '25 23:12 mikomikotaishi

@mikomikotaishi thanks for the additional explanation. I am ok with the following your suggestion.

I have also seen some repositories use bots to push some commits too. Potentially one such bot could be set up to automatically populate the module with new changes each time there is a mismatch. I don't know anything about how to set this up, but I have seen this before and it could potentially be a solution.

We could automatically generate modules/httplib.cppm in a GitHub Actions script when httplib.h is pushed. (I can't accept the way to run a script to update the file manually, since I don't maintain any build systems in this repository. Please see https://github.com/yhirose/cpp-httplib/pull/955#issuecomment-855280013.)

yhirose avatar Dec 09 '25 00:12 yhirose

OK, that makes sense to me. (I don't know anything about how to run GitHub Actions or write scripts for it however, so I'm afraid the most I can do is create a script for this.)

mikomikotaishi avatar Dec 09 '25 01:12 mikomikotaishi

I'm not sure why there were failing workflows as I didn't change anything in the core library

mikomikotaishi avatar Dec 09 '25 02:12 mikomikotaishi

Never mind, it seems the failing CI is happening upstream too.

mikomikotaishi avatar Dec 09 '25 02:12 mikomikotaishi

If the file can be run during the build process (and if the output consists of machine-generated files, it should only run during build time), then the destination directory should be configurable (maybe defaulting to the the current working directory). Even better, if the output is a single file, the script should allow the user to specify the output file itself (full path).

This is because downstreams (like Debian, which is what I maintain the meson build scripts for) may have some requirements on where build products should be stored.

Tachi107 avatar Dec 09 '25 16:12 Tachi107

@Tachi107 CMake needs to know what the output directory is ahead of time to compile the module. How do you propose to solve this?

mikomikotaishi avatar Dec 09 '25 18:12 mikomikotaishi

@yhirose I think there is one possible solution to allow both the directory to be user-specified while still supporting CMake module building, which is probably just to have the Python script generate the CMake file too. I don't know if this is too convoluted or awkward of a design though, so please do tell me your thoughts.

mikomikotaishi avatar Dec 11 '25 07:12 mikomikotaishi

Hi Miko,

On Tue Dec 9, 2025 at 10:46 PM CET, Miko wrote:

@Tachi107 CMake needs to know what the output directory is ahead of time to compile the module. How do you propose to solve this?

You should use CMake's add_custom_command() function to invoke the script and pass it the output file path

https://cmake.org/cmake/help/latest/command/add_custom_command.html

Meson has a similar function, but I can do that myself after this gets merged.

Tachi107 avatar Dec 11 '25 11:12 Tachi107

Is there a reason to try and add module support when the library itself is C++11?

Not to mention, Cmake v3.28 adds built-in module support, which isn't being used either.

sum01 avatar Dec 14 '25 16:12 sum01

This library being C++11 does not require programmes depending on it being C++11.

I support the modules.

Spixmaster avatar Dec 14 '25 16:12 Spixmaster

Indeed, some people who use C++20 and onward may wish to consume the library with the module rather than the header. This is precisely what modules are designed to improve for compilations, as re-including the same large header in multiple locations massively increases build times whereas modules are built only once and then imported/linked.

mikomikotaishi avatar Dec 14 '25 17:12 mikomikotaishi

I understand you can use older standards on newer ones, but I mean trying to add support where it isn't "supported" seems unnecessary.

As for having to re-compile, why not use pre-compiled headers (Cmake v3.16 feature), or the compiled version of this library?

sum01 avatar Dec 14 '25 19:12 sum01

Precompiled headers are not a standard language feature, they are just a compiler trick. Modules provide a very specific API and very specific features for library encapsulation. For example, we may want to avoid including httplib's macros or avoid adding detail/implementation symbols into scope, which overwhelms autocomplete on IDEs with irrelevant symbols.

Even splitting the library into a header/source division still leaves a very large header that gets recompiled each time.

As for the question of "support", this specific wrapper style (a module which includes the header and then re-exports the symbols) is how most libraries built with C++11/14/17 provide module support (such as Boost, Poco, etc.) This is the simplest way to do it to my knowledge, and it's not a hastily tacked-on hack. You can see in other libraries this is how originally header-libraries are wrapped for modules.

mikomikotaishi avatar Dec 14 '25 20:12 mikomikotaishi