workspaces-issues icon indicating copy to clipboard operation
workspaces-issues copied to clipboard
verified publisher icon kasmtech

Unable to upgrade docker

Open Taomyn opened this issue 1 month ago • 2 comments

Describe the bug When I try to upgrade Docker on my Debian 13 machine running KASM 1.18.0 it fails to run

I had KASM 1.17.0 working well on the same machine evenr after upgrading from Debian 12, and the upgrade to 1.18.0 was also fine for months including all updates to Debian until now.

To Reproduce Steps to reproduce the behavior:

  1. Upgrade Docker
  2. Trying to access KASM fails

Screenshots If applicable, add screenshots to help explain your problem.

Workspaces Version 1.18.0

Workspaces Installation Method Single Server

Workspace Server Information (please provide the output of the following commands):

  • uname -a Linux KODOS 6.14.11-4-pve #1 SMP PREEMPT_DYNAMIC PMX 6.14.11-4 (2025-10-10T08:04Z) x86_64 GNU/Linux
  • cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 13 (trixie)" NAME="Debian GNU/Linux" VERSION_ID="13" VERSION="13 (trixie)" VERSION_CODENAME=trixie DEBIAN_VERSION_FULL=13.1 ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
  • sudo docker info Client: Docker Engine - Community Version: 28.5.1 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.29.1 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.38.1 Path: /usr/local/lib/docker/cli-plugins/docker-compose

Server: Containers: 10 Running: 8 Paused: 0 Stopped: 2 Images: 46 Server Version: 28.5.1 Storage Driver: overlay2 Backing Filesystem: zfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: true Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan kasmweb/sidecar:amd64-1.3 macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog CDI spec directories: /etc/cdi /var/run/cdi Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: b98a3aace656320842a23f4a392a33f46af97866 runc version: v1.3.0-0-g4ca628d1 init version: de40ad0 Security Options: seccomp Profile: builtin cgroupns Kernel Version: 6.14.11-4-pve Operating System: Debian GNU/Linux 13 (trixie) OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 6GiB Name: KODOS ID: 98c73f41-edfa-44ea-a4e6-d8f051a6c201 Docker Root Dir: /var/lib/docker Debug Mode: false Experimental: false Insecure Registries: ::1/128 127.0.0.0/8 Live Restore Enabled: false

  • sudo docker ps | grep kasm root@KODOS:~# sudo docker ps | grep kasm 5160ac0eac51 kasmweb/proxy:1.18.0 "/docker-entrypoint.…" 15 minutes ago Up 15 minutes 80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp kasm_proxy 4bf1db518041 kasmweb/rdp-https-gateway:1.18.0 "/opt/rdpgw/rdpgw" 15 minutes ago Up 15 minutes (healthy) kasm_rdp_https_gateway d31365086866 kasmweb/rdp-gateway:1.18.0 "/start.sh" 15 minutes ago Up 15 minutes (healthy) 0.0.0.0:3389->3389/tcp, [::]:3389->3389/tcp kasm_rdp_gateway 37a7599e4c8b kasmweb/agent:1.18.0 "python3 /src/Provis…" 15 minutes ago Up 15 minutes (healthy) 4444/tcp kasm_agent 90269fc3d9de kasmweb/api:1.18.0 "/bin/sh -c /usr/bin…" 15 minutes ago Up 15 minutes (healthy) 8080/tcp kasm_api 52a546e8d327 kasmweb/manager:1.18.0 "python3 /src/api_se…" 15 minutes ago Up 15 minutes (healthy) 8181/tcp kasm_manager 0a961223bc9f kasmweb/postgres:1.18.0 "docker-entrypoint.s…" 15 minutes ago Up 15 minutes (healthy) 5432/tcp kasm_db 5228c8d4780a kasmweb/kasm-guac:1.18.0 "/dockerentrypoint.sh" 15 minutes ago Up 15 minutes (healthy) kasm_guac

Additional context Errors after upgrade:

root@KODOS:~# systemctl status docker.service ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; preset: enabled) Active: active (running) since Fri 2025-11-07 11:19:44 CET; 2min 59s ago Invocation: 2ba5faeb1f1c4edf9dfa15d459fcacec TriggeredBy: ● docker.socket Docs: https://docs.docker.com Main PID: 355 (dockerd) Tasks: 14 Memory: 29.1M (peak: 30.5M) CPU: 296ms CGroup: /system.slice/docker.service └─355 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.535571815+01:00" level=info msg="Docker daemon" commit=89c5e8f containerd-snapshotter=false storage-driver=overlay2 version=28.5.2 Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.535597042+01:00" level=info msg="Initializing buildkit" Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.538146696+01:00" level=info msg="Completed buildkit initialization" Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.543725666+01:00" level=info msg="Daemon has completed initialization" Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.543752606+01:00" level=info msg="API listen on /run/docker.sock" Nov 07 11:19:44 KODOS systemd[1]: Started docker.service - Docker Application Container Engine. Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.808323463+01:00" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stderr Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.808398063+01:00" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.809355624+01:00" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout Nov 07 11:19:44 KODOS dockerd[355]: time="2025-11-07T11:19:44.809377134+01:00" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stderr

Docker version (upgraded):

root@KODOS:~# apt list docker* docker-buildx-plugin/trixie,now 0.29.1-1~debian.13~trixie amd64 [installed,automatic] docker-buildx/stable 0.13.1+ds1-3 amd64 docker-ce-cli/trixie,now 5:28.5.2-1~debian.13~trixie amd64 [installed,automatic] docker-ce-rootless-extras/trixie,now 5:28.5.2-1~debian.13~trixie amd64 [installed,automatic] docker-ce/trixie,now 5:28.5.2-1~debian.13~trixie amd64 [installed] docker-clean/stable 2.0.4-5 all docker-cli/stable 26.1.5+dfsg1-9+b9 amd64 docker-compose-plugin/trixie,now 2.40.3-1~debian.13~trixie amd64 [installed,automatic] docker-compose/stable 2.26.1-4 amd64 docker-doc/stable 26.1.5+dfsg1-9 all docker-model-plugin/trixie 1.0.0-1~debian.13~trixie amd64 docker-registry/stable 2.8.3+ds1-2+b8 amd64 docker.io/stable 26.1.5+dfsg1-9+b9 amd64

Docker service messages before upgrade:

root@KODOS:~# systemctl status docker.service ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; preset: enabled) Active: active (running) since Fri 2025-11-07 11:24:57 CET; 53s ago Invocation: 912430ee487e4b158c4b3238811e2ad1 TriggeredBy: ● docker.socket Docs: https://docs.docker.com Main PID: 387 (dockerd) Tasks: 50 Memory: 89.6M (peak: 93M) CPU: 1.082s CGroup: /system.slice/docker.service ├─ 387 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ├─1489 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.18.0.9 -container-port 443 -use-listen-fd ├─1495 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 443 -container-ip 172.18.0.9 -container-port 443 -use-listen-fd ├─2600 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 3389 -container-ip 172.18.0.7 -container-port 3389 -use-listen-fd └─2608 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 3389 -container-ip 172.18.0.7 -container-port 3389 -use-listen-fd

Nov 07 11:24:57 KODOS dockerd[387]: time="2025-11-07T11:24:57.054043553+01:00" level=info msg="API listen on /run/docker.sock" Nov 07 11:24:57 KODOS systemd[1]: Started docker.service - Docker Application Container Engine. Nov 07 11:24:57 KODOS dockerd[387]: time="2025-11-07T11:24:57.851986042+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:24:58 KODOS dockerd[387]: time="2025-11-07T11:24:58.116643623+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:24:58 KODOS dockerd[387]: time="2025-11-07T11:24:58.424095619+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:24:58 KODOS dockerd[387]: time="2025-11-07T11:24:58.827390012+01:00" level=info msg="ignoring event" container=5bb3bc94e35496bbc828abef276ac211ec8b20f193b485f2a23f26f8b6b674a2 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:24:58 KODOS dockerd[387]: time="2025-11-07T11:24:58.938455242+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:24:59 KODOS dockerd[387]: time="2025-11-07T11:24:59.847526771+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:25:01 KODOS dockerd[387]: time="2025-11-07T11:25:01.559460919+01:00" level=info msg="ignoring event" container=4bf1db518041b54ae4b2331a4cabd162d473b56ab289327e125e3d28a44d3f62 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Nov 07 11:25:07 KODOS dockerd[387]: time="2025-11-07T11:25:07.923416407+01:00" level=info msg="ignoring event" container=d31365086866895fceeabbd084fcc66b0f1984bc80b623b57330b79002cf1be3 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

Docker versions before upgrade:

root@KODOS:~# apt list docker* docker-buildx-plugin/trixie,now 0.29.1-1~debian.13~trixie amd64 [installed,automatic] docker-buildx/stable 0.13.1+ds1-3 amd64 docker-ce-cli/trixie,now 5:28.5.1-1~debian.13~trixie amd64 [installed,automatic] docker-ce-rootless-extras/trixie,now 5:28.5.1-1~debian.13~trixie amd64 [installed,automatic] docker-ce/trixie,now 5:28.5.1-1~debian.13~trixie amd64 [installed] docker-clean/stable 2.0.4-5 all docker-cli/stable 26.1.5+dfsg1-9+b9 amd64 docker-compose-plugin/trixie,now 2.40.3-1~debian.13~trixie amd64 [installed,automatic] docker-compose/stable 2.26.1-4 amd64 docker-doc/stable 26.1.5+dfsg1-9 all docker-model-plugin/trixie 0.1.44-1~debian.13~trixie amd64 docker-registry/stable 2.8.3+ds1-2+b8 amd64 docker.io/stable 26.1.5+dfsg1-9+b9 amd64

Taomyn avatar Nov 07 '25 10:11 Taomyn

Hi there, Can you also include the Kasm service container statuses after the upgrade?

sudo docker ps | grep kasm

Please also include the logs of failing containers (for example, the kasm_proxy container)

sudo docker logs <container_id>

From your docker error logs after the upgrade, it looks like docker is trying to attach a container's stdout/stderr pipes but they no longer exist.

teja-kasm avatar Nov 19 '25 16:11 teja-kasm

This is very strange. I upgraded to 1.80.1 last week without issue, well after I fixed another issue with the script check of docker-compose version (left over manual install of it was still old), but when I tried the update to Docker I had the same issue. I just connected again to get the info you requested but found it was fine. I did a reboot and still worked, but I noticed a few of things:

  • Everything takes much longer to start, which could explain the issue, but also to shut down. Before I could shutdown in a few seconds, now it can take over 30sec before the machine is shut down. I don't think this was the case for my original report as the site would never come on-line, but now with 1.18.1 I just need to wait much longer but it does all start.
  • The kasmweb/kasm-guac:1.18.1 container takes a very long time to start - is this the cause? All the other containers are healthy, but this one stays on healthy,starting for some time.

Taomyn avatar Nov 21 '25 19:11 Taomyn