floooh
How to handle utility libraries that have external dependencies
Today I wanted to add proper font rendering to my project, and thought I would reach for sokol_fontstash.h. My first approach was to try to monkey-patch it into mod_sokol_clib by grabbing it from the dependency within my own project. So I made a local copy of fontstash.h and sokol_fontstash.h, wrapped it all in a sokol_fontstash.c similarly to the c files in this repo, and pulled it into my build file like so:
dep_sokol.module("mod_sokol_clib").addIncludePath(b.path("src/sokol_utils/"));
dep_sokol.module("mod_sokol_clib").addCSourceFile(.{
.file = b.path("src/sokol_utils/sokol_fontstash.c"),
});
This approach did not work, because I could not figure out how to cross the "include boundary" and include C headers from sokol-zig/src/c in my own files.
So I thought that maybe I should just be a good citizen and generate bindings for it using sokol/bindgen. However when I thought about it for half a second, I realized that this would require vendoring fontstash as an upstream dependency somewhere. We could easily do this, but I'm not sure the approach would scale across all of the utils that have upstream deps.
I think the "spirit" of sokol would expect me as an end-user to go obtain fontstash from upstream, as it would with other dependencies (with the exception of cimgui, which is sort of a special case). So I guess I'm asking for some guidance on how to go about this effectively. Alternatively, I'm happy to draft a PR if you have a bindgen-based path forward that you prefer.
I don't have a clean solution for sokol utility headers that have external dependencies. It all involves an amount of hackery that I don't want to commit too ;)
First, there's some things 'vendored' in the sokol repo anyway, this is for building the tests:
https://github.com/floooh/sokol/tree/master/tests/ext
...if we decide that sokol_fontstash.h should be part of the sokol-zig bindings than the cleanest and easiest solution would be to just vendor fontstash.h right in the sokol-zig c subdirectory:
https://github.com/floooh/sokol-zig/tree/master/src/sokol/c
For fontstash that's probably fine since it's just 46 KBytes or so, but for its successor (https://github.com/memononen/Skribidi) that wouldn't work because it has tons of external dependencies.
Another (much cleaner) option would be to maintain separate bindings projects for the utility headers that require external dependencies (like sokol-imgui-zig, sokol-nuklear-zig, sokol-spine-zig, ...) but that would quickly explode into dozens of projects when also taking the other bindings into account, and that would no longer be maintainable by a single person even when the bindings are automatically generated (because there's still some manual work required from time to time).
So long story short: I don't know :)
PS:
Another option might be to only inject the external C deps temporarily for building the bindings during the bindgen phase but then not include them in the Zig package.
...and instead the upstream project would be required to inject those external C dependencies during the build phase - this is essentially like Dear ImGui integration currently works.
E.g. the sokol-zig build.zig would need a new option with_sokol_fontstash like here: https://github.com/floooh/sokol-zig/blob/4b8c06c546ad503d4c765ba5cd9fe8ee3afe3f5b/build.zig#L71
...and in the upstream project the header path to fontstash.h needs to be injected into the C library artifact:
https://github.com/floooh/sokol-zig-imgui-sample/blob/f44d34ba6b4d39540babdc8dd04d8b877d592bd9/build.zig#L30-L31
(allthough tbh, now that I see that I wonder why I don't pass that header path into the sokol dependency like I'm doing with the .with_sokol_imgui setting which would be much cleaner...)
...but this is the sort of build system hackery that I'm not a big fan of... there must be a better way (but with the restriction that it must not require any more maintenance overhead).
I'm a relative newcomer to this project, so take this with a grain of salt, but here's how I see it:
The core vision of sokol is in the tagline: "minimal cross-platform standalone C headers". To me that means that it will give me a minimal platform layer that doesn't pull in other dependencies. I think that's an awesome contract to uphold, both in zig-land and in C-land. Let's never make decisions that add implicit external deps into the zig bindings.
Of course it follows that some users will not want to write their own immediate-mode GUI or font renderer, and want to pull in additional external dependencies. However these dependencies won't be aware of the sokol API, so glue code will be necessary. This is most of what /utils does today. I think this is also an awesome use case to support, but it's a different one. It's making sure that users can hit the ground running with their projects, and have access to enough of the ecosystem to meet their needs.
These are just 2 different concerns, and should be subject to different rules. For the core, Sokol should be self-contained and simple. For utility libraries, it should have some sort of integration with "BYO" C headers from the user. I should be able to write my own sokol_skribidi.h, and pass it into the build system without changing sokol-zig or sokol/bindgen. If sokol continues to grow in popularity, there's a massive list of dependencies people will want to integrate, and you don't want to be stuck holding the bag for all of them. So I think the best investment of effort is finding a really good path to self-serving.
This is sort of trivial in C, but less so in zig. I think the solution will probably be a combination of hardcoding less in the build file and documenting more in the readme. Since it's an open question for you and I have strong opinions, I'd be happy to noodle on this and try to find a solution. I'd just need guidance from you on whether that solution is a good fit for the project.
PS: I guess what I'm trying to say is that for stuff like fontstash, I think it's a fool's errand to try to support all of them in sokol-zig or any of the other language bindings. We should treat it as an external dependency that I'm trying to integrate, and figure out a path for that (even if it means build system hackery).
Yeah spot on, the 'main vision' for the core sokol headers is to not pull in additional dependencies, but for the util headers this doesn't apply (that's also the reason why I didn't want to throw them into the same directory). For C projects that's not much of a problem since the headers are supposed to be copied as needed directly into project source trees.
Maybe a good middle ground is to keep a common sokol package, but at least split up the monolithic C library into multiple libraries (e.g. a core library plus one library per supported util header - and that way the 'build system hackery' would at least be isolated to those util libraries.
For the Zig module it's probably still ok to have a single sokol namespace at the top, since Zig only compiles actually used code.
I think that middle ground makes a lot of sense. The only thing I'd add is probably to have some sort of repeatable "util library template" workflow that users can adopt to reduce friction. That's the piece of connective tissue that we would be on the hook to support.
I'll put the issue into my 'mental backlog'. At some point a cleaner solution is needed since the whole topic of including the util headers in the language bindings has been coming up in the past and will continue to come up, but at the same time there's not much room for more 'manual maintenance overhead' on the language bindings, so the level of automation needs to increase.
Ideally I'd come up with a solution that cleanly maps to all language bindings, not just the Zig bindins.
I don't know if this should be the "blessed" way to handle this problem, but I think I've figured out how I want to handle it:
I like having sokol_fontstash.h part of the same compilation unit as the other sokol sources. I generally want to share the same compilation state between all sokol C deps, and a tight integration makes sense to me. Thankfully the zig build system is mad powerful, so I figured out a way to monkey-patch in additional C sources after the fact:
/// just grab the cflags of the first file we find
fn extract_sokol_cflags(b: *Build) ![]const []const u8 {
const dep_sokol = b.dependency("sokol", .{
.target = target,
.optimize = optimize,
});
const mod_sokol = dep_sokol.module("mod_sokol_clib");
for (mod_sokol.link_objects.items) |lobj| {
if (lobj == .c_source_file) {
return lobj.c_source_file.flags;
}
}
return error.no_csrc_found;
}
fn patch_sokol_with_fontstash(b: *Build) !void {
const dep_sokol = b.dependency("sokol", .{
.target = target,
.optimize = optimize,
});
const mod_sokol = dep_sokol.module("mod_sokol_clib");
mod_sokol.addIncludePath(b.path("src/c_patch/"));
const cflags = try extract_sokol_cflags(b);
mod_sokol.addIncludePath(dep_sokol.path("src/sokol/c"));
mod_sokol.addCSourceFile(.{
.file = b.path("src/c_patch/fontstash.c"),
.flags = cflags,
.language = .c,
});
mod_sokol.link_libc = true;
}
Then I just have a c_patch folder (should probably give it a better name) in my source dir:
├── src
│ ├── c_patch
│ │ ├── fontstash.c
│ │ ├── fontstash.h
│ │ ├── sokol_fontstash.h
│ │ └── stb_truetype.h
│ ├── fontstash-sapp.zig
The only real hack is my extract_sokol_cflags function, which will be brittle to future changes. However I think that if we provide a cflags function or static var within the sokol build.zig function instead, that could be the basis for other monkey-patches like this one.
If you want to see the full implementation, I dumped it here:
https://github.com/axelmagn/zig-fons-example
@axelmagn as an option for bridging the 'include boundary' I used something like this:
mod.addIncludePath(dep_cimgui.path("src")); // or "src-docking" for the docking version provided by dcimgui
mod.addIncludePath(dep_sokol.path("src/sokol/c"));
I didn't go much deeper into that line as I ended up with my own local fork of sokol that I'm playing with at the moment but the above worked when I was using it.
Btw @robboito not sure if it's relevant for your use case, but at least sokol-zig exposes the C library and headers as an artifact:
https://github.com/floooh/sokol-zig/blob/df7a116cfd9348d41f58e48e233b432b0b0c5a8a/build.zig#L318-L323
Those headers can be used from C code by 'linking' your module with the sokol C library artifact, see here https://github.com/floooh/sokol-zig?tab=readme-ov-file#using-sokol-headers-in-c-code.
The dcimgui build.zig is missing this part though (it only makes the C library available as artifact, but not the Dear ImGui headers).