Issues when running Wings with Podman

[RealTriassic]

Current Behavior

Out of curiosity and boredom, I decided to try Podman with Wings after seeing that some people have gotten it working, but ran into a couple issues that prevent this from working smoothly.

• Currently, everytime a server is started, a warning is printed to Wings logs with these contents

WARN: [Nov 18 14:41:12.310] failed to perform on-the-fly update of the server environment error=environment/docker: could not update container: Error response from daemon: Not Found server=89b3e647-dc1a-4a58-85d8-7e35b7d0ab1c

Stacktrace:
Error response from daemon: Not Found
environment/docker: could not update container
github.com/pterodactyl/wings/environment/docker.(*Environment).InSituUpdate
        github.com/pterodactyl/wings/environment/docker/container.go:129
github.com/pterodactyl/wings/server.(*Server).SyncWithEnvironment
        github.com/pterodactyl/wings/server/update.go:51
github.com/pterodactyl/wings/server.(*Server).Sync
        github.com/pterodactyl/wings/server/server.go:200
github.com/pterodactyl/wings/server.(*Server).onBeforeStart
        github.com/pterodactyl/wings/server/power.go:174
github.com/pterodactyl/wings/server.(*Server).HandlePowerAction
        github.com/pterodactyl/wings/server/power.go:131
github.com/pterodactyl/wings/router/websocket.(*Handler).HandleInbound
        github.com/pterodactyl/wings/router/websocket/websocket.go:363
github.com/pterodactyl/wings/router.getServerWebsocket.func3
        github.com/pterodactyl/wings/router/router_server_ws.go:85
runtime.goexit
        runtime/asm_amd64.s:1598

• Additionally, the server will refuse to boot if the server's OOM killer is not enabled with the following error

ERROR: [Nov 18 14:50:35.719] failed to handle server crash error=failed to start server after crash detection: environment/docker: failed to attach to container: environment/docker: error while attaching to container: unable to upgrade to tcp, received 409 server=89b3e647-dc1a-4a58-85d8-7e35b7d0ab1c

Stacktrace:
unable to upgrade to tcp, received 409
environment/docker: error while attaching to container
github.com/pterodactyl/wings/environment/docker.(*Environment).Attach
        github.com/pterodactyl/wings/environment/docker/container.go:61
github.com/pterodactyl/wings/environment/docker.(*Environment).Start
        github.com/pterodactyl/wings/environment/docker/power.go:121
github.com/pterodactyl/wings/server.(*Server).HandlePowerAction
        github.com/pterodactyl/wings/server/power.go:135
github.com/pterodactyl/wings/server.(*Server).handleServerCrash
        github.com/pterodactyl/wings/server/crash.go:90
github.com/pterodactyl/wings/server.(*Server).OnStateChange.func1
        github.com/pterodactyl/wings/server/server.go:324
runtime.goexit
        runtime/asm_amd64.s:1598

environment/docker: failed to attach to container
failed to start server after crash detection
github.com/pterodactyl/wings/server.(*Server).handleServerCrash
        github.com/pterodactyl/wings/server/crash.go:90
github.com/pterodactyl/wings/server.(*Server).OnStateChange.func1
        github.com/pterodactyl/wings/server/server.go:324
runtime.goexit
        runtime/asm_amd64.s:1598

and this is printed to the server's console:

container@pterodactyl~ Error Event [146a8c18-d9b2-4c30-9f94-5f1dee081c9b]: environment/docker: failed to attach to container: environment/docker: error while attaching to container: unable to upgrade to tcp, received 409

Expected Behavior

If actually supported, which I hope it is, Podman should work with Wings without any issues, just like if you were using Docker.

Steps to Reproduce

1. Install Wings using the step-by-step tutorial provided here.
2. Make the following changes to your Wings configuration (config.yml):

• system.enable_log_rotate true -> false
• docker.log_config.type "local" -> ""

There might be solutions that don't require disabling logging, but I am not sure.

Noticed in the Discord that you might need to change these too:

• system.check_permissions_on_boot true -> false
• docker.use_performant_inspect true -> false

I tried with and without the 2 configuration changes above and noticed no difference in functionality, so they may not be needed.

3. Observe the errors I mentioned above in the issue report.

Panel Version

1.11.5

Wings Version

1.11.8

Games and/or Eggs Affected

No response

Docker Image

No response

Error Logs

No response

Is there an existing issue for this?

• [X] I have searched the existing issues before opening this issue.
• [X] I have provided all relevant details, including the specific game and Docker images I am using if this issue is related to running a server.
• [X] I have checked in the Discord server and believe this is a bug with the software, and not a configuration issue with my specific system.

[rhatdan]

You probably have to set the DOCKER_HOST environment variable to point at the podman.sock socket.

[RealTriassic]

You probably have to set the DOCKER_HOST environment variable to point at the podman.sock socket.

This was already done, otherwise servers wouldn't be able to start even with all the workarounds mentioned above.

[matthewpi]

I am aware of the first two parts of this issue (OOM killer and the weird container update errors). I'm unsure why the container update errors out, or what exactly causes it, but everything seems to work fine regardless.

The HTTP 409 error is interesting as it was either fixed in a later Podman release or there was a regression in Wings or the Docker API somewhere.

[leonpano2006]

I am aware of the first two parts of this issue (OOM killer and the weird container update errors). I'm unsure why the container update errors out, or what exactly causes it, but everything seems to work fine regardless.

The HTTP 409 error is interesting as it was either fixed in a later Podman release or there was a regression in Wings or the Docker API somewhere.

Oom killer is caused by CgroupV2 Disabling OOM killer is not possible at all on cgroupV2 crun will give error

[lyra56k]

I'm trying to install Wings with Podman and I'm encountering an issue with the log driver: invalid log driver: invalid argument. Does anyone know how I could remedy this? I can't switch to Docker because I have come to depend on Podman as it came with Rocky Linux 9 and now I work around it.

[lyra56k]

I'm sorry, I somehow failed to see this part:

• system.enable_log_rotate true -> false
• docker.log_config.type "local" -> ""

I fixed that and now am having the same unable to upgrade to tcp. Is this fixable or has someone fixed it yet?

[leonpano2006]

I'm sorry, I somehow failed to see this part:

L Loop~~~~>

• system.enable_log_rotate true -> false

• docker.log_config.type "local" -> ""

I fixed that and now am having the same unable to upgrade to tcp. Is this fixable or has someone fixed it yet?

Is oom killer enabled?(is disable oom killer option disabled? On cgroupV2 oom killer is on as always and no way to disable)

[lyra56k]

Enabling the OOM killer instantly worked, but I'd rather not have that on, because of the possibility of force stopping the server. Is there any other workaround? (It's also possible that I misunderstand OOM killer)

[leonpano2006]

Enabling the OOM killer instantly worked, but I'd rather not have that on, because of the possibility of force stopping the server. Is there any other workaround? (It's also possible that I misunderstand OOM killer)

i said on cgroupv2 disabling oom killer is not possible at all even with normal docker(runc will jump warning about if you disable oom on cgroupv2 and let you run with oom killer, crun will refuse to start in this case)