Build problems with installed libgit2 1.8.0

[0-wiz-0]

This is not the first time similar problems are happening, but this time I'd like to track it down. The current problem is that cargo-c 0.9.31, when built from pkgsrc, doesn't build after the pkgsc libgit2 package is upgraded from 1.7.2 to 1.8.0.

In pkgsrc (and probably most other packaging systems), we'd like to install only one copy of a library so that e.g. fixing security problems is easy - fix the library, rebuild all users, done. When using bundled libgit2, we need to find and fix the problem in all bundled copies of libgit2, which is much more work.

cargo-c 0.9.31 uses libgit2-sys-0.16.2+1.7.2.crate. The build fails with:

...
warning: [email protected]+1.7.2: libgit2/src/libgit2/commit.c:300:5: error: conflicting types for 'git_commit_create'
...
warning: [email protected]+1.7.2: libgit2/src/libgit2/commit.c:939:5: error: conflicting types for 'git_commit_create_buffer'
...
warning: [email protected]+1.7.2: libgit2/src/libgit2/config_backend.h:47:12: error: conflicting types for 'git_config_backend_from_string'
...

(and many more). I suspect that this happens because git2-rs tries to build the bundled libgit2 1.7.2 against the system's 1.8.0 headers.

I see that the documentation says that only newer patch releases of 1.7.2 are supported, so you're already documenting that it won't work. The failure mode when 1.8.0 is installed is hard to diagnose (if you're only interested in cargo-c and just see the build fail).

I have two ideas for this:

• allow newer major versions. Of course you can't predict the future and this might break, but building the included old version against newer headers has already broken a couple of times.
• when building the bundled libgit2 code, make sure to avoid using system-provided headers. (If this is a problem in pkgsrc only, let me know - but it looks like pkgsrc only sets a rust link-arg to fix the rpath.)

Let me know if I got something wrong, or if you have other ideas how to improve the situation. Thank you!

[ehuss]

when building the bundled libgit2 code, make sure to avoid using system-provided headers.

This should be the normal behavior. Perhaps there is an issue with pkg-config or perhaps an include environment variable is set?

Can you show an example using something like docker? For example, with this dockerfile:

FROM ubuntu:latest

WORKDIR /tmp
RUN apt update
RUN apt install -y curl git gcc cmake pkg-config libssl-dev python3
RUN curl -OL https://github.com/libgit2/libgit2/archive/refs/tags/v1.8.0.tar.gz
RUN curl -OL https://github.com/libgit2/libgit2/archive/refs/tags/v1.7.2.tar.gz
RUN tar -xzvf v1.7.2.tar.gz
RUN tar -xzvf v1.8.0.tar.gz

RUN cd libgit2-1.7.2 && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr && cmake --build . -j 8 --target install
RUN pkg-config --modversion libgit2

RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > rustup.sh
RUN sh ./rustup.sh -y --profile minimal
ENV PATH="/root/.cargo/bin:${PATH}"

RUN cargo new foo
WORKDIR /tmp/foo
RUN cargo add [email protected]
RUN echo 'fn main() { println!("{:#?}", git2::Version::get()); }' > src/main.rs
RUN cargo run

RUN cd /tmp/libgit2-1.8.0 && mkdir build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=/usr && cmake --build . -j 8 --target install
RUN pkg-config --modversion libgit2

RUN cargo clean
RUN cargo run

The output for the last commands is:

…
#13 [10/21] RUN pkg-config --modversion libgit2
#13 0.276 1.7.2
#13 DONE 0.3s

#20 [17/21] RUN cargo run
…
#20 4.549      Running `target/debug/foo`
#20 4.551 Version {
#20 4.551     major: 1,
#20 4.551     minor: 7,
#20 4.551     rev: 2,
#20 4.551     crate_version: "0.18.3",
#20 4.551     vendored: false,
#20 4.551     threads: true,
#20 4.551     https: true,
#20 4.551     ssh: false,
#20 4.551     nsec: true,
#20 4.551 }
#20 DONE 4.8s

…

#22 [19/21] RUN pkg-config --modversion libgit2
#22 0.258 1.8.0
#22 DONE 0.3s

#24 [21/21] RUN cargo run
…
#24 10.32      Running `target/debug/foo`
#24 10.32 Version {
#24 10.32     major: 1,
#24 10.32     minor: 7,
#24 10.32     rev: 2,
#24 10.32     crate_version: "0.18.3",
#24 10.32     vendored: true,
#24 10.32     threads: true,
#24 10.32     https: true,
#24 10.32     ssh: true,
#24 10.32     nsec: true,
#24 10.32 }
#24 DONE 10.5s

Here you can see with 1.7.2 installed, it uses the system version (vendored: false). With 1.8 installed it uses the 1.7.2 vendored version.

[0-wiz-0]

Sorry, docker is not native on NetBSD. I looked at the compilation line and it looks like this for me:

cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -O2 -g -fstack-clash-protection -I/usr/pkg/include -I/usr/include -I /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libssh2-sys-361572ac956294e9/out/include -I libssh2/src -I /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libssh2-sys-361572ac956294e9/out/build -I /usr/include -fvisibility=hidden -DHAVE_LONGLONG -DHAVE_SNPRINTF -DHAVE_UNISTD_H -DHAVE_INTTYPES_H -DHAVE_STDLIB_H -DHAVE_SYS_SELECT_H -DHAVE_SYS_SOCKET_H -DHAVE_SYS_IOCTL_H -DHAVE_SYS_TIME_H -DHAVE_SYS_UN_H -DHAVE_O_NONBLOCK -DLIBSSH2_OPENSSL -DHAVE_LIBCRYPT32 -DHAVE_EVP_AES_128_CTR -DHAVE_POLL -DHAVE_GETTIMEOFDAY -DLIBSSH2_DH_GEX_NEW -DLIBSSH2_HAVE_ZLIB -o /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libssh2-sys-361572ac956294e9/out/build/903bc623e643396f-agent.o -c libssh2/src/agent.c
...
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -O2 -g -fstack-clash-protection -I/usr/pkg/include -I/usr/include -I /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libgit2-sys-0e1b24aff7320f06/out/include -I libgit2/src/libgit2 -I libgit2/src/util -I libgit2/deps/http-parser -I libgit2/deps/xdiff -I libgit2/deps/pcre -I /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libssh2-sys-361572ac956294e9/out/include -I /usr/include -fvisibility=hidden -DGIT_REGEX_BUILTIN=1 -DHAVE_STDINT_H=1 -DHAVE_MEMMOVE=1 -DNO_RECURSE=1 -DNEWLINE=10 -DPOSIX_MALLOC_THRESHOLD=10 -DLINK_SIZE=2 -DPARENS_NEST_LIMIT=250 -DMATCH_LIMIT=10000000 -DMATCH_LIMIT_RECURSION=MATCH_LIMIT -DMAX_NAME_SIZE=32 -DMAX_NAME_COUNT=10000 -DSHA1DC_NO_STANDARD_INCLUDES=1 '-DSHA1DC_CUSTOM_INCLUDE_SHA1_C="common.h"' '-DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C="common.h"' -o /scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libgit2-sys-0e1b24aff7320f06/out/build/96639c8a8c534100-openssl.o -c libgit2/src/util/hash/openssl.c

so I suspect that because -I/usr/pkg/include is before -I/scratch/devel/cargo-c/work/cargo-c-0.9.31/target/release/build/libgit2-sys-0e1b24aff7320f06/out/include, the 1.8.0 headers are used instead of the 1.7.2 ones.

Perhaps it is added for another dependency?

Can you try a build where you set CPPFLAGS and/or CFLAGS to the path where libgit2 1.8 is installed on the host system?

(I tried removing pkg-config but openssl-sys uses it for finding openssl.)

[0-wiz-0]

I tried your recipe, manually, and indeed I can reproduce the failure when I use

CFLAGS=-I/usr/pkg/include cargo run

instead of cargo run.