How to add search directories for capnp?

[Biswa96]

In MSYS2/MinGW, the include directory is /mingw64/include/ for x86 and /mingw64/include/ for x86_64. But according to the docs, I have to add -I option in command line to include the search directory. Is it possible to add search directory path at build time? See the discussion at https://github.com/msys2/MINGW-packages/pull/6910.

[kentonv]

Hi @Biswa96,

It sounds like in this environment, you should tell the build system to install to a different prefix than the default /usr/local. You probably want to invoke configure like so:

./configure --prefix=/mingw64

If you do this, the resulting capnp binary will have /mingw64/include in its default include path. Also, setting --prefix this way will cause make install to install Cap'n Proto into /mingw64 in general, which seems like what you want.

If you aren't using the autotools build, you can define the preprocessor symbol CAPNP_INCLUDE_DIR to specify an additional include path. This macro should expand to a string literal. So, you could use a compiler flag like '-DCAPNP_INCLUDE_DIR="/mingw64/include"' -- note that the outer single-quotes are necessary so that the inner double-quotes don't get removed by the shell.

[Biswa96]

Here are something I have tried:

---

CFLAGS+=' -DCAPNP_INCLUDE_DIR=\"/mingw64/include\"' \
CXXFLAGS+=' -DCAPNP_INCLUDE_DIR=\"/mingw64/include\"' \
cmake.exe
// other options here

This compiles but the capnp.exe contains full path of /mingw64/include directory. This causes problem because my installation may not be same with others.

---

CFLAGS+=' -DCAPNP_INCLUDE_DIR="/mingw64/include"' \
CXXFLAGS+=' -DCAPNP_INCLUDE_DIR="/mingw64/include"' \
cmake.exe
// other options here

This causes compiler error.

---

Tried to replace /usr/include with /mingw64/include in src/capnp/compiler/capnp.c++ file. Then compiled it. Compilation went fine. capnp.exe contains /mingw64/include string. But capnp compile fails with error: Import failed: /capnp/c++.capnp.

[kentonv]

I see, so a compile-time constant is not sufficient here, because you want to be able to redistribute the binary and other users may use different installation locations.

I suppose what we really need to do is find the capnp binary itself, then look up ../include relative to it.

This comes with further questions, though. What if the binary is a symlink? Do we look relative to the symlink, or to the executable it points to, or both? What if it's a chain of symlinks?

Also, what's the right way to find the executable's own path? Search for argv[0] using standard PATH rules? Or something more like /proc/self/exe (or Windows equivalent)?

[Biswa96]

I suppose what we really need to do is find the capnp binary itself, then look up ../include relative to it.

Now I am thinking that my question is somewhat msys2 specific. Though searching for ../include is somewhat hardcoded thing.

what's the right way to find the executable's own path?

GetModuleFileName(NULL, buffer, size).

Though I am thinking somewhat like --with-native-system-header-dir option in GCC. I don't know how it is done in GCC. But it may bring some familiarity for everyone.

[kentonv]

GetModuleFileName(NULL, buffer, size)

Right, that's the Windows version of Linux's /proc/self/exe. The problem is, it'll report the final path after symlink resolution. But maybe that's what we want? It seems pretty unclear how symlinks should be handled here.

[LehMaxence]

Just a note that this is also a problem for conda packages.

By default, conda modifies the capnp binary so that the CAPNP_INCLUDE_DIR value set at build time is changed with the correct value at install time but this leads the binary to generating an exception at runtime when the new path is shorter than the placeholder path used at build time. The exception is thrown here https://github.com/capnproto/capnproto/blob/54a702d014515aaed809e83787482fa461cd9eb7/c%2B%2B/src/kj/filesystem.c%2B%2B#L322 because of the added null terminators (see ref https://docs.conda.io/projects/conda-build/en/latest/resources/make-relocatable.html)

[kentonv]

https://docs.conda.io/projects/conda-build/en/latest/resources/make-relocatable.html

That's... terrifying...

[kentonv]

For the record, editing string literals in the capnp compiled binary will not work. KJ does not use NUL terminators to decide a string's length, so if you patch a NUL byte into a string literal, that NUL byte is going to show up in your data. So you'll end up with capnp constructing filenames like /opt/capnp-install-path/include\0\0\0\0\0\0\0\0\0\0\0\0\0/capnp/schema.capnp, which (if not for the aforementioned KJ_REQUIRE) would end up with the OS always trying to open /opt/capnp-install-path/include (a directory) instead of the file you intended.

I do agree that having a way to discover the install location dynamically seems desirable but, as mentioned above, I'm not clear on exactly what the desired behavior is.