amazon-linux-2023 icon indicating copy to clipboard operation
amazon-linux-2023 copied to clipboard

[Bug] - Broken symlink in docker package

Open Venkatesh1505 opened this issue 4 months ago • 9 comments

Describe the bug When docker is installed using yum (or) dnf, it creates the below symlinks.

Image

But, that link is broken, and is not even present.

To Reproduce ls -la /usr/bin | grep containerd

Image

Expected behavior Either the broken symlink should not even be present. If that's absolutely necessary, then an additional symlink should be created. ln -s /usr/bin/containerd-shim-runc-v2 /usr/bin/containerd-shim

Screenshots

Image

Venkatesh1505 avatar Jul 30 '25 09:07 Venkatesh1505

@Venkatesh1505 , thanks for reporting this.

Could you please also specify AL release and Docker version?

alexey-tsvetnov avatar Jul 30 '25 18:07 alexey-tsvetnov

I'm getting the same result

# rpm -q docker
docker-25.0.8-1.amzn2023.0.5.x86_64

# cat /etc/amazon-linux-release
Amazon Linux release 2023.8.20250721 (Amazon Linux)

# ls -la /usr/bin | grep containerd
-rwxr-xr-x 1 root root 61995728 Jun 26 18:35 containerd
-rwxr-xr-x 1 root root 16408616 Jun 26 18:35 containerd-shim-runc-v2
lrwxrwxrwx 1 root root       10 Jun 26 19:47 docker-containerd -> containerd
lrwxrwxrwx 1 root root       15 Jun 26 19:47 docker-containerd-shim -> containerd-shim

# /usr/bin/docker-containerd-shim -v
bash: /usr/bin/docker-containerd-shim: No such file or directory

# /usr/bin/docker-containerd -v
containerd github.com/containerd/containerd/v2 2.0.5 fb4c30d4ede3531652d86197bf3fc9515e5276d9

alexey-tsvetnov avatar Jul 30 '25 18:07 alexey-tsvetnov

I've notified the owning team about the issue. It will be fixed soon.

alexey-tsvetnov avatar Jul 30 '25 18:07 alexey-tsvetnov

@alexey-tsvetnov Here is the OS release requested.

[root@ip-10-56-29-31 ~]# cat /etc/os-release
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.8.20250721"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2029-06-30"

Can we get an ETA on the fix too? TIA

Venkatesh1505 avatar Jul 31 '25 02:07 Venkatesh1505

@alexey-tsvetnov for the owning team:

The problem is also in the previous AL2023 release 2023.8.20250715. My guess is that it's related to the move from containerd-1.7.27 to containerd-2.0.5

$ cat /etc/amazon-linux-release
Amazon Linux release 2023.8.20250715 (Amazon Linux)

$ rpm -q docker
docker-25.0.8-1.amzn2023.0.5.aarch64

$ rpm -q containerd
containerd-2.0.5-1.amzn2023.0.2.aarch64

$ ls -la /usr/bin | grep containerd
-rwxr-xr-x.  1 root root    59956976 Jun 26 18:35 containerd
-rwxr-xr-x.  1 root root    15732144 Jun 26 18:35 containerd-shim-runc-v2
lrwxrwxrwx.  1 root root          10 Jun 26 19:47 docker-containerd -> containerd
lrwxrwxrwx.  1 root root          15 Jun 26 19:47 docker-containerd-shim -> containerd-shim

$ /usr/bin/docker-containerd-shim -v
sh: /usr/bin/docker-containerd-shim: No such file or directory

$ /usr/bin/docker-containerd -v
containerd github.com/containerd/containerd/v2 2.0.5 fb4c30d4ede3531652d86197bf3fc9515e5276d9

Here's a comparison with an earlier release, which does not have the problem

$ cat /etc/amazon-linux-release
Amazon Linux release 2023.7.20250527 (Amazon Linux)

$ rpm -q docker
docker-25.0.8-1.amzn2023.0.4.x86_64

$ rpm -q containerd
containerd-1.7.27-1.amzn2023.0.2.x86_64

$ ls -la /usr/bin | grep containerd
-rwxr-xr-x.  1 root root    61993520 Apr  7 23:41 containerd
-rwxr-xr-x.  1 root root     7723904 Apr  7 23:41 containerd-shim
-rwxr-xr-x.  1 root root    17051656 Apr  7 23:41 containerd-shim-runc-v1
-rwxr-xr-x.  1 root root    21636232 Apr  7 23:41 containerd-shim-runc-v2
lrwxrwxrwx.  1 root root          10 May 16 16:21 docker-containerd -> containerd
lrwxrwxrwx.  1 root root          15 May 16 16:21 docker-containerd-shim -> containerd-shim

$ /usr/bin/docker-containerd-shim -v
containerd-shim
  Version:  1.7.27
  Revision: 05044ec0a9a75232cad458027ca83437aae3f4da
  Go version: go1.23.7

$ /usr/bin/docker-containerd -v
containerd github.com/containerd/containerd 1.7.27 05044ec0a9a75232cad458027ca83437aae3f4da

mwebber avatar Aug 01 '25 10:08 mwebber

According to docker.spec the links should be created as:

# install compatibility symlinks for binaries that moved when runc
# and containerd were split into separate packages
ln -sf ../sbin/runc %{buildroot}%{_bindir}/docker-runc
ln -sf containerd %{buildroot}%{_bindir}/docker-containerd
ln -sf containerd-shim %{buildroot}%{_bindir}/docker-containerd-shim
ln -sf ctr %{buildroot}%{_bindir}/docker-ctr

elsaco avatar Aug 03 '25 02:08 elsaco

A package docker-25.0.8-1.amzn2.0.6 with the fix is available in AL2, https://docs.aws.amazon.com/AL2/latest/relnotes/relnotes-20250902.html :

* Thu Jul 31 2025 Justin Alvarez <[email protected]> - 25.0.8-1.amzn2.0.6
- Rebuild with golang 1.23.11 and remove broken docker-containerd-shim symlink

A similar fixed package docker-25.0.8-1.amzn2023.0.6 for AL2023 is expected to be available in the next regular AL2023 release

alexey-tsvetnov avatar Sep 11 '25 18:09 alexey-tsvetnov

Here's what I see in the latest AL2023 release 2023.8.20250915. It msotly looks ok, except for

$ /usr/bin/docker-containerd-shim -v
sh: /usr/bin/docker-containerd-shim: No such file or directory

Is that expected?

Full details:

$ cat /etc/amazon-linux-release
Amazon Linux release 2023.8.20250915 (Amazon Linux)

$ rpm -q docker
docker-25.0.8-1.amzn2023.0.6.x86_64

$ rpm -q containerd
containerd-2.0.6-1.amzn2023.0.1.x86_64

$ ls -la /usr/bin | grep containerd
-rwxr-xr-x.  1 root root    62002120 Sep  9 19:42 containerd
-rwxr-xr-x.  1 root root    16405376 Sep  9 19:42 containerd-shim-runc-v2
lrwxrwxrwx.  1 root root          10 Aug 26 22:35 docker-containerd -> containerd

$ /usr/bin/docker-containerd-shim -v
sh: /usr/bin/docker-containerd-shim: No such file or directory

$ /usr/bin/docker-containerd -v
containerd github.com/containerd/containerd/v2 2.0.6 991cc3363c290ffd074e069f2b3034c7286ecbe0

mwebber avatar Sep 18 '25 16:09 mwebber

@mwebber , this change (removing the file) was intentional which is, as you correctly guessed, a consequence of moving to containerd v2.

I wonder if you observe any negative impact from this change or have some use case that requires it?

alexey-tsvetnov avatar Sep 18 '25 18:09 alexey-tsvetnov