toolbox: failed to start container fedora-toolbox-RELEASEVER

[mt190502]

Describe the bug

toolbux --log-level debug enter output:

DEBU[0000] Created OCI spec for container 2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec at /home/fedora/.local/share/containers/storage/overlay-containers/2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec/userdata/config.json 
DEBU[0000] /usr/bin/conmon messages will be logged to syslog 
DEBU[0000] running conmon: /usr/bin/conmon               args="[--api-version 1 -c 2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec -u 2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec -r /usr/bin/crun -b /home/fedora/.local/share/containers/storage/overlay-containers/2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec/userdata -p /run/user/1000/containers/overlay-containers/2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec/userdata/pidfile -n fedora-toolbox-37 --exit-dir /run/user/1000/libpod/tmp/exits --full-attach -s -l journald --log-level debug --syslog --conmon-pidfile /run/user/1000/containers/overlay-containers/2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /home/fedora/.local/share/containers/storage --exit-command-arg --runroot --exit-command-arg /run/user/1000/containers --exit-command-arg --log-level --exit-command-arg debug --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/user/1000/libpod/tmp --exit-command-arg --network-config-dir --exit-command-arg  --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /home/fedora/.local/share/containers/storage/volumes --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg --syslog --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec]"
[conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied

DEBU[0000] Received: -1                                 
DEBU[0000] Cleaning up container 2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec 
DEBU[0000] Network is already cleaned up, skipping...   
DEBU[0000] Unmounted container "2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec" 
Error: OCI runtime error: unable to start container "2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec": crun: sd-bus call: Unit libpod-2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec.scope already exists.: File exists

Steps how to reproduce the behaviour

1. Run toolbox create command and create a box.
2. And run the box toolbox enter

Expected behaviour toolbox running smoothly

Actual behaviour toolbox: failed to start container fedora-toolbox-RELEASEVER

Output of toolbox --version (v0.0.90+) toolbox version 0.0.99.3

Toolbox package info (rpm -q toolbox) toolbox-0.0.99.3-7.fc37.x86_64

Output of podman version

Client:       Podman Engine
Version:      4.2.0-rc3
API Version:  4.2.0-rc3
Go Version:   go1.19
Built:        Fri Aug  5 22:53:26 2022
OS/Arch:      linux/amd64

Podman package info (rpm -q podman)

podman-gvproxy-4.2.0~rc3-2.fc37.x86_64
podman-plugins-4.2.0~rc3-2.fc37.x86_64
podman-4.2.0~rc3-2.fc37.x86_64
podman-compose-1.0.3-6.fc37.noarch
podman-docker-4.2.0~rc3-2.fc37.noarch
podman-tests-4.2.0~rc3-2.fc37.x86_64
podman-remote-4.2.0~rc3-2.fc37.x86_64
podman-tui-0.5.0-3.fc37.x86_64

Info about your OS Fedora 37 Rawhide Prerelease

Additional context If I run toolbox with DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus variable, the problem is solved

[debarshiray]

What kind of environment was this that DBUS_SESSION_BUS_ADDRESS wasn't defined? Were you running Toolbx as root? Do you see this:

$ toolbox --log-level debug enter
...
...
Requires org.freedesktop.Flatpak.SessionHelper
...
...

I see that you were using Fedora 37 right after it was branched from Rawhide. So it was still somewhat early in the cycle. Did the problem solve itself? Or are you still experiencing it?

[debarshiray]

Note that on Fedora crun needs D-Bus to work.

Assuming that you were running Toolbx as non-root, then co-relating the crun source with this error:

Error: OCI runtime error: unable to start container "2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec": crun: sd-bus call: Unit libpod-2ce22b06eace2b0eb0f795b95f8f8a84c7e315c09ac4be2a5a703f0b468cd3ec.scope already exists.: File exists

... makes me think that crun failed to connect to the user/session bus due to DBUS_SESSION_BUS_ADDRESS being absent, fell back to using the system bus, and then ran into even more problems.

[debarshiray]

@giuseppe does this look like some bug in Fedora 37 that eventually got fixed?

Given that this bug was filed in August, it doesn't look like it could be https://github.com/containers/podman/issues/16107

[giuseppe]

@debarshiray it could be given the original report says "toolbox-0.0.99.3-7.fc37.x86_64" and the issue happens on Fedora 37

[akdev1l]

it looks fixed from here:

[akdev@halifax ~]$ toolbox create
Image required to create toolbox container.
Download registry.fedoraproject.org/fedora-toolbox:37 (500MB)? [y/N]: y
Created container: fedora-toolbox-37
Enter with: toolbox enter
[akdev@halifax ~]$ toolbox enter 

Welcome to the Toolbox; a container where you can install and run
all your tools.

 - Use DNF in the usual manner to install command line tools.
 - To create a new tools container, run 'toolbox create'.

For more information, see the documentation.

⬢[akdev@toolbox ~]$ ^C
⬢[akdev@toolbox ~]$ 
logout
[akdev@halifax ~]$ toolbox --version
toolbox version 0.0.99.3
[akdev@halifax ~]$ rpm -q toolbox
toolbox-0.0.99.3-7.fc37.x86_64

[mt190502]

I was not using display manager while using Sway. I was opening Sway with this command.

dbus-run-session sway

This gives a random dbus address valid within the session. session only

sway

When I wrote with the command, the dbus address changed to DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus.

[dridi]

I also have a "failed to start container fedora-toolbox-37" situation, but of a different variety:

$ toolbox enter
Error: failed to start container fedora-toolbox-37

$ podman start fedora-toolbox-37
Error: unable to start container "9439c4df3c5ada343b33faad4c7f871bbe49e237256e17c2770bbe22c9546045": crun: mkdir `001`: Permission denied: OCI permission denied

$ distrobox enter fedora-toolbox-37
Container fedora-toolbox-37 is not running.
Starting container fedora-toolbox-37
run this command to follow along:

 podman logs -f fedora-toolbox-37

Error: unable to start container "9439c4df3c5ada343b33faad4c7f871bbe49e237256e17c2770bbe22c9546045": crun: mkdir `001`: Permission denied: OCI permission denied

An error occurred

An error occurred

It should be noted that as of today, there is apparently no proper toolbox image for Fedora 37:

REPOSITORY                                 TAG         IMAGE ID      CREATED        SIZE
registry.fedoraproject.org/fedora-toolbox  37          28543734e061  3 months ago   330 MB

Fedora 37 was released 3 weeks ago, and since I can't start the container in the first place, I can't run a DNF update inside. I still have a working older toolbox container from before I upgraded Fedora from 36 to 37.

[yajo]

I'm also getting this problem since long ago. See attached logs of podman --log-level debug start fedora-toolbox-37: podman-start.log

[alex27riva]

Hi everyone, I have the same issue on Fedora Workstation 40

➜   toolbox enter --log-level debug
DEBU Running as real user ID 1000                 
DEBU Resolved absolute path to the executable as /usr/bin/toolbox 
DEBU Running on a cgroups v2 host                 
DEBU Looking up sub-GID and sub-UID ranges for user username 
DEBU TOOLBOX_PATH is /usr/bin/toolbox             
DEBU Migrating to newer Podman                    
DEBU Toolbox config directory is /home/username/.config/toolbox 
DEBU Current Podman version is 5.1.1              
DEBU Creating runtime directory /run/user/1000/toolbox 
DEBU Old Podman version is 5.1.1                  
DEBU Migration not needed: Podman version 5.1.1 is unchanged 
DEBU Setting up configuration                     
DEBU Setting up configuration: file /home/username/.config/containers/toolbox.conf not found 
DEBU Resolving container and image names          
DEBU Container: ''                                
DEBU Distribution (CLI): ''                       
DEBU Image (CLI): ''                              
DEBU Release (CLI): ''                            
DEBU Resolved container and image names           
DEBU Container: 'fedora-toolbox-40'               
DEBU Image: 'fedora-toolbox:40'                   
DEBU Release: '40'                                
DEBU Resolving container and image names          
DEBU Container: ''                                
DEBU Distribution (CLI): ''                       
DEBU Image (CLI): ''                              
DEBU Release (CLI): ''                            
DEBU Resolved container and image names           
DEBU Container: 'fedora-toolbox-40'               
DEBU Image: 'fedora-toolbox:40'                   
DEBU Release: '40'                                
DEBU Checking if container fedora-toolbox-40 exists 
DEBU Inspecting mounts of container fedora-toolbox-40 
DEBU Starting container fedora-toolbox-40         
Error: failed to start container fedora-toolbox-40