OpenBSD support

[margaretjoanmiller]

I would absolutely love it if rustup had OpenBSD support. We're still pretty similar to NetBSD and rust is available on OpenBSD so it shouldn't be too difficult to port.

[rbtcollins]

So - start by building rustup yourself - if that works, and that channel metadata for rust distributions has everything you need, then this ticket becomes 'please make rustup binaries available for download'. If it doesn't build / run properly, we can help you with that, and if the channel metadata or the rust distributions are missing things we can steer you in the right direction.

[margaretjoanmiller]

Hmm tried running cargo run --release and got

error: Pre-checks for host and toolchain failed: Provided host 'x86_64-unknown-openbsd' couldn't be converted to partial triple
If you are unsure of suitable values, the 'stable' toolchain is the default.
Valid host triples look something like: x86_64-unknown-openbsd

[rbtcollins]

Ok, so thats the starting point - you'll need to teach the host triple detection about openbsd. I don't have the source handy just now, but as you said NetBSD was similar I suggest grepping for NetBSD in the source.

[kinnison]

OpenBSD is all tier 3 at this time, which means the required dockers for rustup to build will not be available since they are not produced for tier 3 platforms as I understand it. If I'm wrong then in theory we could add support to our CI to test these platforms at least, even if we're not ready to release yet.

[mneumann]

@kinnison I use DragonFly, which is not supported as well. You should still be able to install rustup locally!

Clone rustup source and run (if it fails to build, see my branch):

cargo run -- --default-host x86_64-unknown-freebsd -y

This will download and install the FreeBSD binaries. We don't need them, but rustup can't just install itself without downloading anything. IMHO, would be great if this would be fixed in rustup.

If you would now run $HOME/.cargo/bin/rustc, you would run the FreeBSD binaries... not good. That's why we have to tell rustup about the rust toolchain from ports (I assume this is in /usr/local on OpenBSD):

$HOME/.cargo/bin/rustup toolchain link ports-toolchain /usr/local
$HOME/.cargo/bin/rustup default ports-toolchain

If you now run $HOME/.cargo/bin/rustc, you run the OpenBSD rustc.

If you can, don't install rustc etc. into /usr/local. Install it into some directory which is not in your PATH.

Hope this helps! Good luck!

[mneumann]

@kinnison If you want to use rustup for cross-compilation: "rustup target add ..." does not work when the ports-toolchain is activated. The workaround is to switch to the freebsd toolchain, install the target, and switch back.

[reynoldsbd]

Using @mneumann's guidance, I was able to get rustup to build and install on OpenBSD. Thanks!

Unfortunately, the binary fails immediately due to rust-lang/rust#60560. I am going to reach out to the OpenBSD port maintainer to try and find a solution here. In the meantime, one can work around by setting a shell alias to rustup's absolute path:

$ alias rustup=$HOME/.cargo/bin/rustup

@kinnison regarding OpenBSD being tier-3, are you aware of any ongoing conversations to change that? I would love to get involved here :)

[semarie]

OpenBSD is a 3-tier platform for several reasons:

• it is required to bootstrap OpenBSD rustc binaries from Linux. I am unsure it could be easily doable as OpenBSD linker is specific (and I am unsure how many things has been upstreamed to ld.lld or GNU ld)
• OpenBSD provides no compatibility between OS versions. It means rustup will need to target a specific stable OS version and kept up-to-date at 6 month cycle to provide useful things. For OpenBSD users running -current (which is common), it means no rustup support (as libc could change in breaking way easily).

[rogerxxxx]

Be aware. Rust bootstrap requires SSE2, most Pentium3 (i686) and earlier CPU's do not have SSE2 CPU register, resulting in segmentation faults. Can workaround this by building the rust bootstrap yourself, while most distributions now rely upon rust prebuilts SSE2 bootstraps. I've yet to see any explanation within the past year as to this requirement.

[shaneutt]

I'm interested in potentially working on this, can someone help me put together a TODO list of what it will take to get OpenBSD support added?

[semarie]

@shaneutt the first steps would be:

• [ ] be able to build working OpenBSD binaries from Linux (without that, I doubt there will be any official support for OpenBSD). it should need some patches in llvm/libc++/libc++abi
• [ ] decide which OpenBSD version should be targeted and be prepared to update it regulary (release cycle is 6 months based). it is important: else you will just add frustration for people who might want to use it or add work for others people

next, there are unsupported things like @reynoldsbd mentioned (env::current_exe for example).

[shaneutt]

Thanks @semarie 👍

I'll do some digging in the codebase.

[workingjubilee]

@rustbot label: +O-bsd

[shaneutt]

I did end up digging into this, but I've had to drop it due to other life priorities. The patches to llvm/libc are the bigger painpoints there in terms of the amount of time that needs to be dedicated to this.

[hargoniX]

I'd like to look into this as well, @shaneutt did you end up pushing some results into some repository? Or could you share your insights into the problem in general maybe?

[shaneutt]

I'd like to look into this as well, @shaneutt did you end up pushing some results into some repository? Or could you share your insights into the problem in general maybe?

I only made it as far as coming to understand how the FreeBSD support was added as a frame of reference. I can recommend you doing the same, but beyond that I have nothing meaningful to contribute.

[xmawja]

i found this script may help you out.

[grinsingularity]

I use OpenBSD v7.2 and trying to build https://github.com/mimblewimble/grin

[hargoniX]

It seems people are making progress with respect to OpenBSD in Github CI: https://github.com/mario-campos/emulate could this be used to properly compile rustc binaries in the CI for our purposes?

[shaneutt]

It seems people are making progress with respect to OpenBSD in Github CI: https://github.com/mario-campos/emulate could this be used to properly compile rustc binaries in the CI for our purposes?

So this seems really cool, but as I understand it the bigger barrier to entry for official support are the fairly large sized things pointed out in https://github.com/rust-lang/rustup/issues/2168#issuecomment-815507459.

That said, maybe "unofficial support" is something we could aim for? If I'm understanding things correctly, we might be able to use something like this to create an unofficial rustup repository and generate and host OpenBSD Rust builds as an interim solution?

[workingjubilee]

-Zbuild-std now works for some targets, so that could be used as an initial basis once the compiler issues are sorted.

[sbourdeauducq]

Note that many of the limitations of the Rust compiler from the OpenBSD ports (or any other "stable" Rust package) are arbitrary and self-inflicted, and can often be worked around by setting the magic environment variable RUSTC_BOOTSTRAP=1. This is much faster than trying to get rustup to work.

[sbourdeauducq]

Example building an ARM MCU embedded project with rustc from OpenBSD ports:

1. get rid of another arbitrary, frustrating check in cargo-xbuild:

diff --git a/src/lib.rs b/src/lib.rs
index 204ad8f..9c61e39 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -157,28 +157,11 @@ pub fn build(args: Args, command_name: &str, crate_config: Option<Config>) -> Re
         })
     })?;
 
-    // We can't build sysroot with stable or beta due to unstable features
     let sysroot = rustc::sysroot(verbose)?;
-    let src = match meta.channel {
-        Channel::Dev => rustc::Src::from_env().ok_or(anyhow!(
+    let src = rustc::Src::from_env().ok_or(anyhow!(
             "The XARGO_RUST_SRC env variable must be set and point to the \
-             Rust source directory when working with the 'dev' channel",
-        ))?,
-        Channel::Nightly => {
-            if let Some(src) = rustc::Src::from_env() {
-                src
-            } else {
-                sysroot.src()?
-            }
-        }
-        Channel::Stable | Channel::Beta => {
-            bail!(
-                "The sysroot can't be built for the {:?} channel. \
-                 Switch to nightly.",
-                meta.channel
-            );
-        }
-    };
+             Rust source directory",
+        ))?;
 
     let cmode = if let Some(triple) = args.target() {
         if triple == meta.host {

2. cargo install --path . to install the modified cargo-xbuild
3. get a copy of the Rust sources that corresponds to the installed compiler binary:

> rustc --version
rustc 1.72.1 (d5c2e9c34 2023-09-13) (built from a source tarball)
> git clone --depth=1 --recurse-submodules --shallow-submodules https://github.com/rust-lang/rust --branch 1.72.1

4. set up environment

> export RUSTC_BOOTSTRAP=1
> export XARGO_RUST_SRC=/path_to_checkout/library

5. enjoy

> cargo xbuild

Of course none of this would be necessary if the Rust policy was to treat users like adults.