AMP icon indicating copy to clipboard operation
AMP copied to clipboard

AMP doesn't stop instances/applications properly

Open BroOtti opened this issue 1 year ago • 14 comments

[Update 11.09.2025]

Version v2.6.2.8, built 09/09/2025 16:36 fixes the graceful stop for instances running in docker. Instances not running in docker are still killed instead of gracefully shutdown.


Bug Report

System Information

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Key                  ┃ Value                                 ┃
┣━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
┃ Operating System     ┃ Linux - Debian GNU/Linux 12 on x86_64 ┃
┃ Product              ┃ AMP 'Callisto' v2.5.0.4 (Mainline)    ┃
┃ Virtualization       ┃ ProxmoxLXC                            ┃
┃ Application          ┃ Application Deployment                ┃
┃ Module               ┃ ADSModule                             ┃
┃ Running in Container ┃ No                                    ┃
┃ Current State        ┃ Indeterminate                         ┃
┗━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

I confirm:

  • [x] that I have searched for an existing bug report for this issue.
  • [x] that I am using the latest available version of AMP.
  • [x] that my operating system is up-to-date.

Symptoms

Trying to stop an instance through ampinstmgr stop or all instances through ampinstmgr -o doesn't work as expected. It just hangs after the first instance, killing only the first instance. (Only happens for instances running in docker) Example:

  CubeCoders AMP  💻amp@amp  📁~  ampinstmgr stop Minecraft
[Info] AMP Instance Manager v2.5.0.4 built 03/04/2024 18:20
[Info] Stream: Mainline / Release - built by CUBECODERS/buildbot on CCL-DEV
[Info] Stopping Instance: 'Minecraft01'
[Info] Stopping instance Minecraft01...
AMP_Minecraft01
[Info] Error response from daemon: cannot kill container: AMP_Minecraft01: No such container: AMP_Minecraft01

I need to CTRL+C to end ampinstmgr. What happens, the container gets killed without a proper shutdown of the application. There is no shutdown visible in the minecraft logs. I expect the application to shutdown properly on instance shutdown to prevent data loss. That happens unrelated to docker.

Reproduction

  1. Create multiple instances with docker enabled. (In my test case, I chose a minecraft instance)
  2. Manage the instances, update and start them.
  3. Try stopping the instances through ampinstmgr -o
  4. Notice only one instance gets killed and the other keep running.
  5. Check the application log if it was stopped properly. (In my test case with minecraft, the server was just killed and not stopped properly)

BroOtti avatar Apr 04 '24 08:04 BroOtti

First posted on discord: https://ptb.discord.com/channels/266012086423912458/1225357651848724531 Currently, no extra info at discord. I will keep it here.

BroOtti avatar Apr 04 '24 08:04 BroOtti

+1 for this

Error response from daemon: cannot kill container

used ampinstmgr upgradeall

Nazgile94 avatar Apr 04 '24 16:04 Nazgile94

I can confirm this issue, same for me.

Sh4d0w0561 avatar Apr 06 '24 07:04 Sh4d0w0561

To give this issue an update before we do experiments..

Still no graceful shutdown with and without docker on version v2.6.2.6, built 04/08/2025 23:38

BroOtti avatar Sep 03 '25 11:09 BroOtti

To keep track here...

docker exec AMP_Factorio02 su -l amp -c "/usr/bin/pkill -SIGINT -f /AMP/AMP_Linux_$(arch)" pkill -SIGINT -f /home/amp/.ampdata/instances/test01/AMP_Linux_$(arch)

These commands (Thanks to Mike) work to do a graceful shutdown for the specified instance. AMP just needs to implement it.

BroOtti avatar Sep 03 '25 12:09 BroOtti

There's actually a more concise version:

docker exec -u amp AMP_INSTANCENAME pkill -SIGINT -f /AMP/AMP_Linux_$(arch)

Which is what I've implemented for the next update.

PhonicUK avatar Sep 03 '25 14:09 PhonicUK

Version v2.6.2.8, built 09/09/2025 16:36 fixes the graceful stop for instances running in docker. Instances not running in docker are still killed instead of gracefully shutdown.

BroOtti avatar Sep 11 '25 08:09 BroOtti

Mike doesn't believe you 🤣

Greelan avatar Sep 11 '25 10:09 Greelan

Ok, I just noticed the graceful shutdown isn't working if I restart the whole server xD Maybe the service need an update? Will look into it later. Maybe also time to setup a test server again instead messing up with my friends xD


Edit: Turns out, on server shutdown docker.service was stopped before ampinstmgr.service which caused the instances to be killed instead of a graceful shutdown xD

BroOtti avatar Sep 12 '25 07:09 BroOtti

Version v2.6.2.8, built 11/09/2025 15:04 Docker instances are gracefully stopped with ampinstmgr stop and ampinstmgr stopall. On shutdown docker instances are gracefully stopped if the ampinstmgr.service is adjusted. Instances running without docker are still killed and not gracefully stopped.

BroOtti avatar Sep 12 '25 19:09 BroOtti

Version v2.6.2.8, built 11/09/2025 15:04 Docker instances are gracefully stopped with ampinstmgr stop and ampinstmgr stopall. On shutdown docker instances are gracefully stopped if the ampinstmgr.service is adjusted. Instances running without docker are still killed and not gracefully stopped.

Was it doing this before the patch? In my linked issue with Factorio, I set those instances to non-docker mode and they were gracefully shutting down whether I shutdown/rebooted from the terminal, or the hypervisor via qemu-guest-agent. It was only docker mode that was not gracefully terminating the instances on my end.

dreamdenizen avatar Sep 12 '25 19:09 dreamdenizen

Version v2.6.2.8, built 11/09/2025 15:04 Docker instances are gracefully stopped with ampinstmgr stop and ampinstmgr stopall. On shutdown docker instances are gracefully stopped if the ampinstmgr.service is adjusted. Instances running without docker are still killed and not gracefully stopped.

Was it doing this before the patch? In my linked issue with Factorio, I set those instances to non-docker mode and they were gracefully shutting down whether I shutdown/rebooted from the terminal, or the hypervisor via qemu-guest-agent. It was only docker mode that was not gracefully terminating the instances on my end.

Factorio is also killed if not run in docker. Which OS do you use? I'm testing with Debian 13. Can you share the latest AMP Log after starting the server and then stopping the instance?

BroOtti avatar Sep 12 '25 19:09 BroOtti

Version: 2.6.2.8 - 20250916.2 OS: Debian 13.1 amd64

Did some more testing tonight. I'm still seeing the original behavior I reported in Issue: https://github.com/CubeCoders/AMP/issues/1354 When in non-container mode, on system shutdown, the factorio process saves the game and gracefully exits. If I switch it to Docker mode, it just exits, without saving. Switching it back to non-container mode restores expected behavior.

dreamdenizen avatar Sep 20 '25 07:09 dreamdenizen

Update with instances on v2.6.3, built 24/09/2025 23:08 and ampinstmgr on v2.6.3, built 24/09/2025 23:10 for Debian 13 Instances with docker enabled are gracefully shutdown. Instances without docker are killed.

BroOtti avatar Sep 25 '25 09:09 BroOtti