cross-rs
containerd + nerdctl support
Checklist
- [X] I've looked through the issues and pull requests for similar request
- [ ] This feature could be solved with a custom image (optional)
Describe your request
Feature request: Support containerd + nerdctl additionally to dockerd (moby) and podman.
Currently, if cross-rs is run on a machine with containerd + nerdctl the following error pops up:
> cross build --target aarch64-unknown-linux-gnu
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
It would be great, if other container runtimes would be detected and used. If this is not much work and can be done by someone unfamiliar with the code of the cross project, feel free to point me to the right direction.
Describe why this would be a good inclusion for cross
With the commercialisation of docker desktop, many developers switched to other solutions like Rancher Desktop. Per default containerd + nerdctl is used instead of docker to run containers.
The whole Kubernetes ecosystems seems to move rapidly into the direction of containerd, so I think it would be great if cross-rs could use that, too.
We should already support this, can you run with -v to show how nerdctl was invoked and what failed.
We detect what is used here and we query some more information from the container engine. I think that's what is going wrong
Sure, here is the output:
cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /Users/X/.rd/bin/docker
+ /Users/X/.rd/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=X' --rm --user 1707927531:1282186731 -v /Users/X/.xargo:/xargo:z -v /Users/X/.cargo:/cargo:z -v /cargo/bin -v /Users/X/repos/gdcli:/project:z -v /Users/X/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /Users/X/repos/gdcli/target:/target:z -w /project -i -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target aarch64-unknown-linux-gnu -v'
docker: error during connect: Head "http://%2Fvar%2Frun%2Fdocker.sock/_ping": EOF.
See 'docker run --help'.
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
Some more information about the system:
> nerdctl -v
nerdctl version 1.7.6
> sw_vers
ProductName: macOS
ProductVersion: 14.6
BuildVersion: 23G80
> cross -V
cross 0.2.5
I use Rancher Desktop 1.14.1 to manage containerd and nerdctl.
Can you try updating cross to the newest master? cargo install cross --git https://github.com/cross-rs/cross
Hi! I updated to the latest cross version from git.
> cross -V
cross 0.2.5 (7b79041 2024-07-11)
> cross build --target aarch64-unknown-linux-gnu
Error:
0: could not get os and arch
1: `docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'` failed with exit status: 1
Location:
src/errors.rs:147
Error:
0: could not get os and arch
1: `docker version -f '{{ .Client.Os }},,,{{ .Client.Arch }}'` failed with exit status: 1
Stderr:
error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/version": EOF
can you show me docker version
is bin/docker actually docker or nerdctl? There might be an issue here were your docker binary is incorrectly actually docker and not containerd via nerdctl
if nerdctl is what is needed, you could use the env-var CROSS_CONTAINER_ENGINE=nerdctl
I think we are getting closer to the issue.
> docker version
Client:
Version: 26.1.0-rd
API version: 1.45
Go version: go1.21.9
Git commit: cca8e72
Built: Tue Apr 23 16:25:03 2024
OS/Arch: darwin/arm64
Context: default
error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/version": EOF
> which docker
/Users/X/.rd/bin/docker
> which nerdctl
/Users/X/.rd/bin/nerdctl
It seems that Rancher installs nerdctl and containerd as the default but additionally docker + moby as it is possible to switch to those, too. In the nerdctl default mode, docker cli is available but docker is not running, so there is no sock to connect to. If cross checks if docker exits and tries to use it, it fails.
Telling cross to use nerdctl explicitly seems the right solution, if that's the issue. Unfortunately setting the env. var. leads to another error:
> export CROSS_CONTAINER_ENGINE=nerdctl
> cross build --target aarch64-unknown-linux-gnu
FATA[0000] unknown flag: --userns
Error: exit status 1
Btw, thanks for being super responsive @Emilgardis!
ugh, --userns is not documented, thus nerdctl doesn't implement it :D (docker/docs#7071)
we should remove userns when engine is nerdctl, however for now you can do CROSS_CONTAINER_USER_NAMESPACE=none
Yeah! That starts to pull the image!
But I'm really sorry to bother you with the next issue :D
> export CROSS_CONTAINER_ENGINE=nerdctl
> export CROSS_CONTAINER_USER_NAMESPACE=none
> cross build --target aarch64-unknown-linux-gnu
ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main: resolved |++++++++++++++++++++++++++++++++++++++|
index-sha256:9808d7b7563a6334042c00555e473d0e0be46a171e3426301030118807280c86: done |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:168fa652629cedcd9549e85258be8f83fa008625b187c004d6ea439cf16f6a41: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:09d507265d4d68195761f92e4386ff2c59996d7b25bc8bab4a433f52d684f527: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ec1ec5f628bd5c711a177303506e91f3cabeea17a9a99a4560427e0c6d3d36ac: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:2437fb9edcccd88fae58518f0a173701c3b76a003748306ff2f14ca587373bd4: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9ea8908f47652b59b8055316d9c0e16b365e2b5cee15d3efcb79e2957e3e7cad: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:372e12503c26c04efde3b5ac5c8b48169444c25c4a29a51d71fe20f001bfb8b3: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:56bc9cd530688a4a1dd2205e9f3b92df9518c68abb9449e8c1bbe532d3436a46: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f26e5a1a56aab880851c8acd969160322adc38286581b5b41eaf6e1763f57512: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9f301d56d553b39fbc538e99b834b0d34dd6c718bb9d8875f343b7ba3328e548: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4983db9bd2821e903a8521249618fcc7564fad14ecb1296f19c579fc852a1f6f: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d2daf07ccb455dbc9baf0b58b4a9afba02f297dcfa4fcc88430de62b40ae6f48: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:20ccf0dd482e3e2110f45d1249699211cb58a318e30916aec3ceca967d075ffb: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:3822ee7993bdb2659e69053703b2e39146854328b049d4c0ee3d7e903b019479: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:47c0e240713a26fdde7313857795b4fe3233b8dbdde1d06ca324fdd84f5eb58a: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9e5154ac9f3d3847a95c498c53c815fe2f43b04c49328589c068bfeff4791d89: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c730e5e04ef635503a334bc044f2e35eeb046092fbf1f82888dece5028755b46: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ed0a30f1e47185d95b906b75da08bba15ae2251cd4a490c52f11e22bd7ebd58a: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c14f9f387e8ec83528cdfa29aec28da41abb5086ba244fcacf0fe0771b535474: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f95d806fbd629f8a113d0132987e2ac9b0734aecbbbfaa4f7f683c630b8193ba: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:75e06a0297aa12adad112e95cfbb1bd7cefae376d927c37b901797614dd3e8fa: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:33779e87ffe4697ca66d852bbe0b5bebd68eff670aacb654554beb7df3a07768: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 52.7s total: 551.3 (10.5 MiB/s)
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
FATA[0052] invalid name "cross-stable-x86_64-unknown-linux-gnu-96bfb-9b00956e5-aarch64-unknown-linux-gnu-00a65-1722935715050": identifier "cross-stable-x86_64-unknown-linux-gnu-96bfb-9b00956e5-aarch64-unknown-linux-gnu-00a65-1722935715050" greater than maximum length (76 characters): invalid argument
Error: exit status 1
meh, that seems arbitrary.
you can rename the image pre-build uses by manually doing what it does...
#Cross.toml
[build]
image = "my-new-name"
dockerfile = "./Dockerfile"
#Dockerfile
ARG CROSS_BASE_IMAGE
FROM $CROSS_BASE_IMAGE
RUN <do stuff here>
kindof a hack but oh well