Implement a FreeBSD syscall layer

[andrewrk]

source: https://lobste.rs/s/yjd59n/crt_free_2023_tips_tricks#c_viakfm

The task is to change this to false for FreeBSD: https://github.com/ziglang/zig/blob/282cb5ee5d75b4de2f215479b0c5531750908608/lib/std/target.zig#L398-L413

And then go implement syscalls in the standard library via inline assembly like we already do for Linux rather than relying on libc for this target.

This is great news, hurray! We no longer need to build libc in order to cross-compile zig code for this target.

[davidchisnall]

All of the system call stubs in FreeBSD are machine generated from syscalls.master. If you want them as inline assembly, you can probably reuse the code.

Note that the syscall calling convention is a little bit interesting on FreeBSD. Arguments are passed in the same location that they are for normal function calls (just replace the call instruction with syscall), with the exception of x8-64, where the SysV ABI defines a different register for one of the arguments (the trampoline is then just a register-register move and a syscall). The return values are more complex. On sensible architectures the error state is defined by the carry bit. If the carry bit is set, the return value register contains the errno value, if it is not set then it contains the result. This lets you branch on carry to detect failure. I believe MIPS and RISC-V use the second return register.

[andrewrk]

Thanks for the tips!

[RealDeuce]

So, it's worth noting though that syscalls.master is version-specific and will change across FreeBSD major versions.
For example, swapoff() changed from 13 to 14:

424	AUE_SWAPOFF	COMPAT13 {
		int swapoff(
		    _In_z_ const char *name
		);
	}


582	AUE_SWAPOFF	STD {
		int swapoff(
		    _In_z_ const char *name,
		    u_int flags,
		);
	}

COMPATXX are optionally compiled in (IIRC, the current supported releases are included by default). I've assumed this is why FreeBSD has used libc for syscalls.

[andrewrk]

I'm feeling a bit confused about whether there is a stable syscall ABI or not. What can an application developer expect if they provide a binary to a FreeBSD OS user?

[RealDeuce]

Just looked... it appears GENERIC includes COMPATXXX all the way back to 4 when it was introduced, so less of a concern than I thought.

[RealDeuce]

So my understanding is that basically, the ABI is stable, but it's possible for the user to exclude parts of it.

I just discovered that by default, the whole thing is included... I had previously thought old syscalls were removed from the default kernel, but that's not true.

[RealDeuce]

So an application developer can expect that the binary will work for the version it was compiled for, and every version after that unless the user has built a custom kernel that removes backward compatibility.

[RealDeuce]

So the only real "gotcha" is making sure that the syscalls.master used is from the oldest supported FreeBSD, to avoid using syscalls from the future.

[mikdusan]

For our purposes, syscalls.master is lacking some information. Specifically, we need to map a syscall to the semver of FreeBSD where it was introduced. The good news is information can be gleaned from the freebsd.src repository's tagged versions of syscalls.master.

Using syscall swapoff as an example, the latest release 13.2.0 syscalls.master provides this information:

1. syscall_424 swapoff is marked with COMPAT13 meaning after 13.0.0 this became backwards compatible.
2. syscall_582 swapoff is marked with STD meaning this is current for 13.2.0. But it doesn't tell us exactly when it was introduced. We can guess it was introduced at or after 13.0.0 but "at or after" is not good enough.

Manually searching history we learn:

1. syscall_424 swapoff was introduced in 5.1.0. Interestingly, the manpage suggests 5.0.0 was the introduction.
2. syscall_582 swapoff was introduced in 13.1.0.

This looks very promising indeed!

[RealDeuce]

I'm not sure why the semver for each syscall would be needed unless the plan is to have each FreeBSD release from 4.x on be a unique target... not that I'm opposed to that, it just seems like a lot of unnecessary work, and makes targeting "FreeBSD" more complex than I would expect it to be.

[igalic]

I believe MIPS and RISC-V use the second return register.

MIPS is no longer supported.

[dch]

@emaste can you offer any further wisdom here? If zig can rely on using oldest supported syscall tables then it could generate FreeBSD compatible binaries directly from any other OS/architecture which is very desirable.

• https://ziglang.org/learn/overview/#zig-competes-with-c-instead-of-depending-on-it

It would help FreeBSD to be a tier 1 target for zig compiler instead of a tier 2.

• https://ziglang.org/download/0.11.0/release-notes.html#Support-Table

[emaste]

So the only real "gotcha" is making sure that the syscalls.master used is from the oldest supported FreeBSD, to avoid using syscalls from the future.

Yes, it's unfortunate that you had to hunt for this information, but this is correct. We may eventually remove COMPAT entries for very old FreeBSD versions from the default kernel configuration, but I expect that wouldn't happen for a minimum of a couple of decades after EOL.

I'm not sure why the semver for each syscall would be needed

If you're bypassing libc the libc stubs and associated symver isn't important. 12.x is the oldest supported FreeBSD release so that may be what you want to target, but it is EOL in a few more months and 13.x may be preferable.

Rust and Go have similar issues. Discussions in https://github.com/rust-lang/rust/issues/89058 and https://github.com/golang/go/issues/48164 have a lot of information that is relevant here.

Manually searching history we learn: syscall_424 swapoff was introduced in 5.1.0. Interestingly, the manpage suggests 5.0.0 was the introduction.

Hmm, let's see. swapoff was added by https://github.com/freebsd/freebsd-src/commit/92da00bb245b0398f04fdd966ce001599060f40c and indeed I see that is between 5.0 and 5.1. Fixed in https://github.com/freebsd/freebsd-src/commit/b15f6400376a90d3b00aa3ac00666f683f975376

All of the system call stubs in FreeBSD are machine generated from syscalls.master.

Generated by https://github.com/freebsd/freebsd-src/blob/main/sys/tools/makesyscalls.lua

[mchoo7]

From what I read so far, it seems that we don't need to do bring libc headers.

If so, the last checkbox of #1759 can be marked done and #2876 can be closed

[nektro]

freebsd will still have to be added to wiki/Updating-libc once -lc cross compilation is supported, which is being tracked in #2876

[alexrp]

The task is to change this to false for FreeBSD:

This was done in #23835. The check is now here:

https://github.com/ziglang/zig/blob/cc1475c91da6005d72192b426e8b9ec6db7a3f74/src/Compilation/Config.zig#L324-L349

The check can be removed once the std.os.freebsd syscall layer has been implemented.