Cross building x86_64 and arm64 on a arm64 machine inside docker does not work

[ares-b]

Checklist

• [x] I've looked through the issues and pull requests for similar reports

Describe your issue

Hey,

I'm running Gitea + Gitea runner on a arm64 VM hosted on Oracle Cloud.

I'm getting this error:

info: downloading component 'rust-src'
info: installing component 'rust-src'
exec /usr/bin/sh: exec format error

What target(s) are you cross-compiling for?

No response

Which operating system is the host (e.g computer cross is on) running?

• [ ] macOS
• [ ] Windows
• [x] Linux / BSD
• [ ] other OS (specify in description)

What architecture is the host?

• [x] x86_64 / AMD64
• [ ] arm32
• [x] arm64 (including Mac M1)

What container engine is cross using?

• [x] docker
• [ ] podman
• [ ] other container engine (specify in description)

cross version

0.2.5

Example

name: Rust CI

on:
  push:
    branches:
      - release-next
      - main

env:
  CARGO_TERM_COLOR: always

jobs:
  rust-ci:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        target: 
          - aarch64-unknown-linux-gnu
          - x86_64-unknown-linux-gnu

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      
      - uses: dtolnay/rust-toolchain@stable
        with:
          toolchain: stable
          components: clippy, rustfmt
          target: ${{ matrix.target }}

      - name: Install cross
        run: cargo install cross --git https://github.com/cross-rs/cross.git
        shell: bash

      - name: Cross version
        run: cross --version
        shell: bash 
      
      - name: Check cargo fmt
        run: cargo fmt --all -- --check
        shell: bash

      - name: Build
        run: cross build --target ${{ matrix.target }}
        shell: bash

      - name: Run Tests
        run: cross test --target ${{ matrix.target }}
        shell: bash
      
      - name: Run Clippy
        run: cross clippy --target ${{ matrix.target }} -- -D warnings
        shell: bash

Additional information / notes

cross 0.2.5 (c7dee4d 2025-02-12) info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu' info: latest update on 2025-01-30, rust version 1.84.1 (e71f9a9a9 2025-01-27) info: downloading component 'cargo' info: downloading component 'rust-std' info: downloading component 'rustc' info: installing component 'cargo' info: installing component 'rust-std' info: installing component 'rustc' stable-x86_64-unknown-linux-gnu installed - (error reading rustc version) info: checking for self-update info: downloading component 'rust-std' for 'aarch64-unknown-linux-gnu' info: installing component 'rust-std' for 'aarch64-unknown-linux-gnu' [cross] note: Falling back to cargo on the host. cargo 1.84.1 (66221abde 2024-11-19)

[Emilgardis]

can you please share the entire log while using -v

[ares-b]

Hi @Emilgardis of course.

When Installing cross with cargo install cross --git https://github.com/cross-rs/cross.git:

With aarch64-unknown-linux-gnu :

root@aaa34604cb02:/tmp/toto# cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ /usr/bin/docker
+ /usr/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/bin/docker info -f {{.SecurityOptions}}
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker run --userns host --platform linux/amd64 -e 'XARGO_HOME=/root/.xargo' -e 'CARGO_HOME=/root/.cargo' -e 'CROSS_RUST_SYSROOT=/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' -e 'CROSS_RUSTC_MAJOR_VERSION=1' -e 'CROSS_RUSTC_MINOR_VERSION=84' -e 'CROSS_RUSTC_PATCH_VERSION=1' --name cross-stable-x86_64-unknown-linux-gnu-53e1a-e71f9a9a9-aarch64-unknown-linux-gnu-4049c-1739646983394 --rm --user 0:0 -v /root/.xargo:/root/.xargo:z -v /root/.cargo:/root/.cargo:z -v /root/.cargo/bin -v /tmp/toto:/tmp/toto:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -v /tmp/toto/target:/target:z -w /tmp/toto -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main sh -c 'PATH="$PATH":"/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin" cargo build --target aarch64-unknown-linux-gnu -v'
exec /usr/bin/sh: exec format error
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

With x86_64-unknown-linux-gnu:

root@aaa34604cb02:/tmp/toto# cross build --target x86_64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform x86_64-unknown-linux-gnu
+ rustc --print sysroot
+ /usr/bin/docker
+ /usr/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/bin/docker info -f {{.SecurityOptions}}
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker run --userns host --platform linux/amd64 -e 'XARGO_HOME=/root/.xargo' -e 'CARGO_HOME=/root/.cargo' -e 'CROSS_RUST_SYSROOT=/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' -e 'CROSS_RUSTC_MAJOR_VERSION=1' -e 'CROSS_RUSTC_MINOR_VERSION=84' -e 'CROSS_RUSTC_PATCH_VERSION=1' --name cross-stable-x86_64-unknown-linux-gnu-53e1a-e71f9a9a9-x86_64-unknown-linux-gnu-4049c-1739647024959 --rm --user 0:0 -v /root/.xargo:/root/.xargo:z -v /root/.cargo:/root/.cargo:z -v /root/.cargo/bin -v /tmp/toto:/tmp/toto:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -v /tmp/toto/target:/target:z -w /tmp/toto -t ghcr.io/cross-rs/x86_64-unknown-linux-gnu:main sh -c 'PATH="$PATH":"/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin" cargo build --target x86_64-unknown-linux-gnu -v'
exec /usr/bin/sh: exec format error
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

When installing cross with cargo install cross: With aarch64-unknown-linux-gnu :

root@42232e8d9150:/tmp/toto# cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup --verbose component add rust-src --toolchain stable-x86_64-unknown-linux-gnu
verbose: read metadata version: '12'
info: downloading component 'rust-src'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2025-01-30/rust-src-1.84.1.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: installing component 'rust-src'
verbose: creating temp directory: /root/.rustup/tmp/eu7ymvtdol5w2jtn_dir
verbose: creating temp file: /root/.rustup/tmp/gdeqfutxg5vkexwh_file
verbose: creating temp file: /root/.rustup/tmp/h4iaqq55duunn6uf_file
verbose: deleted temp directory: /root/.rustup/tmp/eu7ymvtdol5w2jtn_dir
verbose: creating temp file: /root/.rustup/tmp/dd7_9ulaexzdst50_file
verbose: creating temp file: /root/.rustup/tmp/9e15oc7rhrbw2w7b_file
verbose: deleted temp file: /root/.rustup/tmp/gdeqfutxg5vkexwh_file
verbose: deleted temp file: /root/.rustup/tmp/h4iaqq55duunn6uf_file
verbose: deleted temp file: /root/.rustup/tmp/dd7_9ulaexzdst50_file
verbose: deleted temp file: /root/.rustup/tmp/9e15oc7rhrbw2w7b_file
+ /usr/bin/docker
+ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' --rm --user 0:0 -v /root/.xargo:/xargo:z -v /root/.cargo:/cargo:z -v /cargo/bin -v /tmp/toto:/project:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /tmp/toto/target:/target:z -w /project -i -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target aarch64-unknown-linux-gnu -v'
Unable to find image 'ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5' locally
0.2.5: Pulling from cross-rs/aarch64-unknown-linux-gnu
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

Running CROSS_CONTAINER_OPTS="--platform linux/amd64" cross build --target x86_64-unknown-linux-gnu -v :

root@42232e8d9150:/tmp/toto# CROSS_CONTAINER_OPTS="--platform linux/amd64" cross build --target x86_64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform x86_64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker
+ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' --platform linux/amd64 --rm --user 0:0 -v /root/.xargo:/xargo:z -v /root/.cargo:/cargo:z -v /cargo/bin -v /tmp/toto:/project:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /tmp/toto/target:/target:z -w /project -i -t ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target x86_64-unknown-linux-gnu -v'
exec /bin/sh: exec format error
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
root@42232e8d9150:/tmp/toto#

Tries cross 0.2.4 and still have the same error

[Emilgardis]

oh, this is on a aarch64 machine, should've noticed that immediately... You'll need to install docker run --privileged --rm tonistiigi/binfmt --install amd64

[ares-b]

oh, this is on a aarch64 machine, should've noticed that immediately... You'll need to install docker run --privileged --rm tonistiigi/binfmt --install amd64

I've got a different error after running the command:

root@26f30fe6323c:/tmp/toto# cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker
+ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' --rm --user 0:0 -v /root/.xargo:/xargo:z -v /root/.cargo:/cargo:z -v /cargo/bin -v /tmp/toto:/project:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /tmp/toto/target:/target:z -w /project -i -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target aarch64-unknown-linux-gnu -v'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
sh: 1: cargo: not found
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu




root@26f30fe6323c:/tmp/toto# cross build --target x86_64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform x86_64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker
+ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=root' --rm --user 0:0 -v /root/.xargo:/xargo:z -v /root/.cargo:/cargo:z -v /cargo/bin -v /tmp/toto:/project:z -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /tmp/toto/target:/target:z -w /project -i -t ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target x86_64-unknown-linux-gnu -v'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
sh: 1: cargo: not found
+ rustup component list --toolchain stable-x86_64-unknown-linux-gn

[Emilgardis]

is cargo installed in another fs not well supported by docker maybe?

Can you try

docker run --rm --userns host --platform linux/amd64 -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -it ubuntu:20.04 ls /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu

if that doesn't work, try upgrading cross to the main branch (very much doubt it'll fix it though) cargo install cross --git https://github.com/cross-rs/cross

[ares-b]

is cargo installed in another fs not well supported by docker maybe?

Can you try

docker run --rm --userns host --platform linux/amd64 -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -it ubuntu:20.04 ls /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu

if that doesn't work, try upgrading cross to the main branch (very much doubt it'll fix it though) cargo install cross --git https://github.com/cross-rs/cross

ubuntu@instance20250211204829:~$ sudo docker run --rm --userns host --platform linux/amd64 -v /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -it ubuntu:20.04 ls /root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu
Unable to find image 'ubuntu:20.04' locally
20.04: Pulling from library/ubuntu
d9802f032d67: Already exists
Digest: sha256:8e5c4f0285ecbb4ead070431d29b576a530d3166df73ec44affc1cd27555141b
Status: Downloaded newer image for ubuntu:20.04
ubuntu@instance20250211204829:~$

Cargo isn't installed anywhere. I'm actually spinning up a fresh container running ubuntu:latest for my tests. Already tried installing cross from the main branch :/

root@12716a7b2202:/tmp/toto# cross build --target aarch64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2025-01-30, rust version 1.84.1 (e71f9a9a9 2025-01-27)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
 26.7 MiB /  26.7 MiB (100 %)  15.2 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 70.6 MiB /  70.6 MiB (100 %)  16.6 MiB/s in  4s ETA:  0s

  stable-x86_64-unknown-linux-gnu installed - (error reading rustc version)

info: checking for self-update
info: downloading component 'rust-std' for 'aarch64-unknown-linux-gnu'
info: installing component 'rust-std' for 'aarch64-unknown-linux-gnu'
 29.4 MiB /  29.4 MiB (100 %)  15.2 MiB/s in  1s ETA:  0s
sh: 1: cargo: not found

[Emilgardis]

For container in container you need to use https://github.com/cross-rs/cross/wiki/FAQ#docker-in-docker

[ares-b]

For container in container you need to use https://github.com/cross-rs/cross/wiki/FAQ#docker-in-docker

Yes, I'm mounting the docker socket. That's how I'm running the container where I'm trying my builds:

 sudo docker run -it -v /var/run/docker.sock:/var/run/docker.sock gitea/runner-images:ubuntu-20.04 bash

I tried setting the env variable CROSS_CONTAINER_IN_CONTAINER=true before installing cargo and cross and have the same error :/

root@01bde907b1aa:/tmp/toto# cross build --target aarch64-unknown-linux-gnu
error: DEPRECATED: future versions of rustup will require --force-non-host to install a non-host toolchain.
warning: toolchain 'stable-x86_64-unknown-linux-gnu' may not be able to run on this system.
warning: If you meant to build software to target that platform, perhaps try `rustup target add x86_64-unknown-linux-gnu` instead?
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2025-01-30, rust version 1.84.1 (e71f9a9a9 2025-01-27)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
 26.7 MiB /  26.7 MiB (100 %)  15.4 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 70.6 MiB /  70.6 MiB (100 %)  16.7 MiB/s in  4s ETA:  0s

  stable-x86_64-unknown-linux-gnu installed - (error reading rustc version)

info: checking for self-update
info: downloading component 'rust-std' for 'aarch64-unknown-linux-gnu'
info: installing component 'rust-std' for 'aarch64-unknown-linux-gnu'
 29.4 MiB /  29.4 MiB (100 %)  14.9 MiB/s in  1s ETA:  0s
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
sh: 1: cargo: not found


root@01bde907b1aa:/tmp/toto# cross build --target aarch64-unknown-linux-gnu --force-non-host
info: downloading component 'rust-src'
info: installing component 'rust-src'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
sh: 1: cargo: not found

[Emilgardis]

you need to have the env var set while running cross. Is it set?

the warning about --force-non-host is from rustup and not related to your issue. Please use cross from the main branch while continuing this troubleshooting, that error shouldn't exist on the main branch, same goes for the warning about requested image's platform not matching.

[ares-b]

you need to have the env var set while running cross. Is it set?

the warning about --force-non-host is from rustup and not related to your issue. Please use cross from the main branch while continuing this troubleshooting, that error shouldn't exist on the main branch, same goes for the warning about requested image's platform not matching.

I've tried using the main branch and now I've got this error:

root@7b98d07fb683:/tmp/toto# CROSS_CONTAINER_IN_CONTAINER=true cross build --target aarch64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2025-01-30, rust version 1.84.1 (e71f9a9a9 2025-01-27)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
 26.7 MiB /  26.7 MiB (100 %)  15.1 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 70.6 MiB /  70.6 MiB (100 %)  16.4 MiB/s in  4s ETA:  0s

  stable-x86_64-unknown-linux-gnu installed - (error reading rustc version)

info: checking for self-update
info: downloading component 'rust-std' for 'aarch64-unknown-linux-gnu'
info: installing component 'rust-std' for 'aarch64-unknown-linux-gnu'
 29.4 MiB /  29.4 MiB (100 %)  15.2 MiB/s in  1s ETA:  0s
error: process didn't exit successfully: `rustc -vV` (signal: 11, SIGSEGV: invalid memory reference)
--- stderr
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

[Emilgardis]

Yeah, im not sure how to fix this. Somethibg is seriously messed up it seems with emulation. The problem is that we need to emulate x86_64 because the images are all currently x86_64 only. You could craft your own image that has aarch64 support but its not super simple. See https://github.com/cross-rs/cross/issues/751

[Emilgardis]

Can you run the build again but with -v please

[ares-b]

Can you run the build again but with -v please

Sure:

root@7b98d07fb683:/tmp/toto# CROSS_CONTAINER_IN_CONTAINER=true cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ /usr/bin/docker
+ /usr/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/bin/docker info -f {{.SecurityOptions}}
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/docker inspect 7b98d07fb683
+ /usr/bin/docker run --userns host --platform linux/amd64 -e 'XARGO_HOME=/root/.xargo' -e 'CARGO_HOME=/root/.cargo' -e 'CROSS_RUST_SYSROOT=/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e CROSS_CONTAINER_IN_CONTAINER -e TERM -e 'USER=root' -e 'CROSS_RUSTC_MAJOR_VERSION=1' -e 'CROSS_RUSTC_MINOR_VERSION=84' -e 'CROSS_RUSTC_PATCH_VERSION=1' --name cross-stable-x86_64-unknown-linux-gnu-53e1a-e71f9a9a9-aarch64-unknown-linux-gnu-4049c-1739706620094 --rm --user 0:0 -v /var/lib/docker/overlay2/71c2fc4944136266084696a9dd37fe0de395df1fca90cf8d3e19eaa74671bcf3/merged/root/.xargo:/root/.xargo:z -v /var/lib/docker/overlay2/71c2fc4944136266084696a9dd37fe0de395df1fca90cf8d3e19eaa74671bcf3/merged/root/.cargo:/root/.cargo:z -v /root/.cargo/bin -v /var/lib/docker/overlay2/71c2fc4944136266084696a9dd37fe0de395df1fca90cf8d3e19eaa74671bcf3/merged/tmp/toto:/tmp/toto:z -v /var/lib/docker/overlay2/71c2fc4944136266084696a9dd37fe0de395df1fca90cf8d3e19eaa74671bcf3/merged/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -v /var/lib/docker/overlay2/71c2fc4944136266084696a9dd37fe0de395df1fca90cf8d3e19eaa74671bcf3/merged/tmp/toto/target:/target:z -w /tmp/toto -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main sh -c 'PATH="$PATH":"/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin" cargo build --target aarch64-unknown-linux-gnu -v'
error: process didn't exit successfully: `rustc -vV` (signal: 11, SIGSEGV: invalid memory reference)
--- stderr
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

[ares-b]

Yeah, im not sure how to fix this. Somethibg is seriously messed up it seems with emulation. The problem is that we need to emulate x86_64 because the images are all currently x86_64 only. You could craft your own image that has aarch64 support but its not super simple. See #751

It worked with the builded image, guess I'll have to build those images before running my rust CI

[Emilgardis]

Awesome, hopefully soon we'll distribute prebuilt images for aarch64, do you think we can close this issue as the immediate problem is resolved?

[ares-b]

Awesome, hopefully soon we'll distribute prebuilt images for aarch64, do you think we can close this issue as the immediate problem is resolved?

Yup we can close it !

[lilith]

Running cross on host aarch64-unknown-linux-gnu with target aarch64-unknown-linux-gnu or aarch64-unknown-linux-musl (in order to lower glibc version) also crashes, might be related:

Run /home/runner/work/_temp/cross +stable test --target aarch64-unknown-linux-musl --locked --release --features=mimalloc
/home/runner/work/_temp/8afe309c-c719-42af-9f2d-90968f16aeef.sh: line 1: /home/runner/work/_temp/cross: cannot execute binary file: Exec format error

Using via https://github.com/houseabsolute/actions-rust-cross/issues/39

[autarch]

This looks like an error from trying to execute cross, not a bug in cross. There's no cross release for arm64, so I wonder if my action is just installing the x86-64 binary. It shouldn't, but it could be.

But it looks like arm64 runners are now available for FOSS project - https://github.blog/changelog/2025-01-16-linux-arm64-hosted-runners-now-available-for-free-in-public-repositories-public-preview/

I will give it a try.

[autarch]

Yeah, I can confirm this is a bug in action.

[autarch]

Ok, there may also be an issue with cross when compiling for x86-64 on an arm host. Here's a log: https://github.com/houseabsolute/actions-rust-cross/actions/runs/13364309080/job/37319067651

You can see that cross did execute and it invoked Docker, then something blew up. This run used the latest commit from the cross main branch, c7dee4d008475ce1c140773cbcd6078f4b86c2aa.

[Emilgardis]

Ok, there may also be an issue with cross when compiling for x86-64 on an arm host. Here's a log: https://github.com/houseabsolute/actions-rust-cross/actions/runs/13364309080/job/37319067651

You can see that cross did execute and it invoked Docker, then something blew up. This run used the latest commit from the cross main branch, c7dee4d.

Have you installed docker run --privileged --rm tonistiigi/binfmt --install amd64 here? I can't see it in the logs. You can use https://github.com/docker/setup-qemu-action to do it also

[autarch]

Ah, indeed, I did not have that step. I added it and now there's a different error:

Status: Downloaded newer image for ghcr.io/cross-rs/x86_64-unknown-linux-gnu:main
error: process didn't exit successfully: `rustc -vV` (signal: 11, SIGSEGV: invalid memory reference)
--- stderr
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Error: Process completed with exit code 101.

https://github.com/houseabsolute/actions-rust-cross/actions/runs/13374148428/job/37349382757#step:4:898

[Emilgardis]

Ugh, same error, this has worked in the past on my mac m1, will have to investigate more... Ill reopen this issue.

[Emilgardis]

To troubleshoot this, we need to figure out why rustc gets garbled and identify when it happens

[Emilgardis]

As a workaround, you can craft an aarch64 image to use as detailed in an earlier comment https://github.com/cross-rs/cross/issues/1628#issuecomment-2661394533

[autarch]

That linked comment doesn't really explain to me what sort of custom aarch64 image I would use. It seems like #751 sort of covers this, but even after reading through that one I'm at a loss as to how to make this work.

But FWIW, I don't need to do this for any project of my own other than trying to support this in actions-rust-cross. I think @lilith is trying to get this working via my action for some sort of real project.

[lilith]

Yeah, I publish prebuilt binaries for imazen/imageflow (dll/dylib/so) and would love to use cross centos to make them work on more linux versions (curse glibc), but I can't get cross to run on ubuntu-22.04-arm and build arm64 binaries, nor run cross on x64 and build arm64 binaries.

[Emilgardis]

nor run cross on x64 and build arm64 binaries.

can you please open a new issue or discussion for this, that's a different issue 👍🏼

[garasubo]

I have the similar issue and investigated the reason. I used cross on Ubuntu 24.04 on WSL on arm64 Windows. Cross doesn't have x64 based images, so we cannot use cross without emulation. I also tried to use my custom images, which support multi platform using docker buildx, but it uses the x64 based image instead of arm64 one. The reason is it decides the platform based on provided images, so the platform is always fallback into x64 (e.g. https://github.com/cross-rs/cross/blob/c7dee4d008475ce1c140773cbcd6078f4b86c2aa/src/lib.rs#L814).

I think this will resolve in #1630 .

[Emilgardis]

#1630 will not solve this issue immediately, it does not include building and pushing the images. And it also doesnt explain why theres suddenly issues with using our x86_64 images on aarch64 docker.