glibc 2.27 or older: fcntl64 not found, but zig's glibc headers refer it

[motiejus]

TLDR: zig is using "too new" glibc headers, which sometimes references undefined symbols. This fails compilation for at least sqlite and libuv when older glibc is selected, because it redefines fcntl to fcntl64, which is present only in newer glibcs.

main.c

#define _FILE_OFFSET_BITS 64
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>

int main() {
    printf("address to fcntl: %p\n", fcntl);
}

This fails when target is glibc 2.27 or older:

$ zig cc --target=x86_64-linux-gnu.2.27  main.c
ld.lld: error: undefined symbol: fcntl64
>>> referenced by main.c
>>>               /home/motiejus/.cache/zig/o/921a4d8978936f8450e53a6103470e2c/main.o:(main)
>>> did you mean: fcntl64@GLIBC_2.28
>>> defined in: /home/motiejus/.cache/zig/o/6dd7f9446c261fd02c47b1aad02ab90b/libc.so.6

And works with glibc 2.28+:

$ zig cc --target=x86_64-linux-gnu.2.28  main.c
$ ./a.out 
address to fcntl: 0x7f46a328e330

This task gives a small reproducible test case; the problem was well explained in https://github.com/ziglang/zig/issues/5882#issuecomment-748974297 , includes a workaround (for x86_64 only though). While the workaround works, it may be nicer if zig provided headers of the requested version, and make this problem go away?

[motiejus]

Also related: 39083c31a550ed80f369f60d35791e98904b8096 and https://patchwork.sourceware.org/project/glibc/patch/[email protected]/

[motiejus]

if https://patchwork.sourceware.org/project/glibc/patch/[email protected]/ gets merged, I will attempt do to an equivalent thing for fcntl64 for glibc <= 2.27.

[etanol]

The solution provided in 5882 didn't fully work for me when building CPython. Therefore, I tried something similar to 39083c31a550ed80f369f60d35791e98904b8096:

--- zig_linux_x86.orig/lib/libc/include/generic-glibc/fcntl.h	2022-02-15 03:47:43.000000000 +0100
+++ zig_linux_x86.custom/lib/libc/include/generic-glibc/fcntl.h	2022-06-22 12:50:07.530393034 +0200
@@ -173,7 +173,7 @@
    This function is a cancellation point and therefore not marked with
    __THROW.  */
 #ifndef __USE_TIME_BITS64
-# ifndef __USE_FILE_OFFSET64
+# if (__GLIBC__ == 2 && __GLIBC_MINOR__ < 28) || !defined(__USE_FILE_OFFSET64)
 extern int fcntl (int __fd, int __cmd, ...);
 # else
 #  ifdef __REDIRECT

And the linking issue disappeared. However, the resulting binary behaves erratically (for instance, when the mkdir syscall fails, errno returns 0); but I can't confirm any relationship with the linking problem described in this issue.

[etanol]

This issue not only impacts fcntl64. For instance, memfd_create was added in 2.27; but when linking in a system with an older glibc (e.g. Amazon Linux 2), memfd_create fails to resolve because it is missing in the installed Glibc but present in Zig's bundled headers.

I would like to know if Glibc headers are included via automation or are hand tuned sometimes. Because the preprocessor conditionals may be the easiest way to solve this. In the case of memfd_create, surrounding it by something like #if (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 27) || __GLIBC__ > 2.

[motiejus]

This issue not only impacts fcntl64. For instance, memfd_create was added in 2.27; but when linking in a system with an older glibc (e.g. Amazon Linux 2), memfd_create fails to resolve because it is missing in the installed Glibc but present in Zig's bundled headers.

Good point.

I would like to know if Glibc headers are included via automation or are hand tuned sometimes. Because the preprocessor conditionals may be the easiest way to solve this. In the case of memfd_create, surrounding it by something like #if (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 27) || __GLIBC__ > 2.

There is a precedent in 39083c31a550ed80f369f60d35791e98904b8096 ; but it's dangerous and it would be best to keep it at zero, since that will make glibc header updates error prone.

I know @marler8997 started working on a proper solution to glibc headers, and this issue is high in our wishlist (but to my knowledge nobody in ZSF has prioritized it yet).

[etanol]

While this issue is being considered. Is there a way to tell the Zig C driver to use the system's libc headers instead of the bundled ones?

[motiejus]

While this issue is being considered. Is there a way to tell the Zig C driver to use the system's libc headers instead of the bundled ones?

Yes. Just don't pass --target:

main.c

#define _FILE_OFFSET_BITS 64
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>

int main() {
    printf("address to fcntl: %p\n", fcntl);
}

Compile:

$ strace -f -e openat zig cc main.c |& grep fcntl.h
[pid 2239029] openat(AT_FDCWD, "/usr/include/fcntl.h", O_RDONLY|O_NOCTTY|O_LARGEFILE|O_CLOEXEC) = 29
[pid 2239029] openat(AT_FDCWD, "/usr/include/x86_64-linux-gnu/bits/fcntl.h", O_RDONLY|O_NOCTTY|O_LARGEFILE|O_CLOEXEC) = 29

[xxxbxxx]

This workaround seems to "work-for-me":

redirect_fnctl64_hack.zig:

// work around glibc headers >= 2.28 no linking against older runtime library
// more info: https://microeducate.tech/how-to-force-linkage-to-older-libc-fcntl-instead-of-fcntl64/

extern fn fcntl() callconv(.Naked) i32;

pub export fn fcntl_zig_trampoline() callconv(.Naked) noreturn {
    const builtin = @import("builtin");
    if (builtin.target.isGnuLibC()) {
        const ver = builtin.os.version_range.linux.glibc;
        if (comptime ver.order(.{ .major = 2, .minor = 28, .patch = 0 }) == .lt) {
            @export(fcntl_zig_trampoline, .{ .name = "fcntl64", .linkage = .Weak });
            if (builtin.target.cpu.arch == .x86_64) {
                asm volatile (
                    \\ jmp fcntl
                );
            } else {
                @compileError("TODO");
            }
        }
    }

    unreachable;
}

and add this to the build.zig script:

        const obj = b.addObject("fnctl64_hack", "redirect_fnctl64_hack.zig");
        obj.setTarget(target);
        obj.setBuildMode(.ReleaseFast);
        exe.addObject(obj);

(or I guess it's similarly possible with other build systems by using zig build-obj and including the .o file to the linker command)

[motiejus]

I would like to know if Glibc headers are included via automation or are hand tuned sometimes. Because the preprocessor conditionals may be the easiest way to solve this.

I discussed this offline with @andrewrk and we agreed to do header ifdefs until universal-headers project is merged.

#15101 adds header conditionals to fix fcntl64 and a few more symbols from resolv.h.