cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard
verified publisher icon apache

Unable to delete or cleanup a network whose state is struck in Implementing

Open kiranchavala opened this issue 2 weeks ago • 0 comments

problem

Unable to delete or cleanup a network whose state is struck in Implementing

versions

ACS 4.22 and older versions

The steps to reproduce the bug

  1. Have a network struck in implementing state
Image
  1. Click on delete network

Exception "failed to delete network"

Image

Logs

root@acs420:~# cat   /var/log/cloudstack/management/management-server.log |grep -i "logid:82f88545"
2025-12-12 08:49:54,434 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Executing AsyncJob {"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","cmdInfo":"{\"response\":\"json\",\"ctxUserId\":\"2\",\"sessionkey\":\"pbLHDerttlq72JFt8rB7UYkO_30\",\"httpmethod\":\"POST\",\"ctxStartEventId\":\"333\",\"id\":\"b32b5f06-a846-41f3-bd49-b485839386d3\",\"ctxDetails\":\"{\\\"interface com.cloud.network.Network\\\":\\\"b32b5f06-a846-41f3-bd49-b485839386d3\\\"}\",\"ctxAccountId\":\"2\",\"uuid\":\"b32b5f06-a846-41f3-bd49-b485839386d3\",\"cmdEventType\":\"NETWORK.DELETE\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":119,"initMsid":206863097656310,"instanceId":207,"instanceType":"Network","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"82f88545-13af-46be-8669-bd6d7bb01604"}
2025-12-12 08:49:54,436 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119, ctx-d445e199]) (logid:82f88545) Account [Account [{"accountName":"admin","id":2,"uuid":"69c635a2-d5e9-11f0-a8f5-bc241173f3f6"}]] has access to resource.
2025-12-12 08:49:54,449 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-2:[ctx-720b416c, job-119, ctx-d445e199]) (logid:82f88545) Lock is acquired for network Network {"id": 207, "name": "test-user", "uuid": "b32b5f06-a846-41f3-bd49-b485839386d3", "networkofferingid": 10} as a part of network shutdown
2025-12-12 08:49:54,449 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-2:[ctx-720b416c, job-119, ctx-d445e199]) (logid:82f88545) Network is not implemented: Network {"id": 207, "name": "test-user", "uuid": "b32b5f06-a846-41f3-bd49-b485839386d3", "networkofferingid": 10}
2025-12-12 08:49:54,456 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-2:[ctx-720b416c, job-119, ctx-d445e199]) (logid:82f88545) Lock is released for network Network {"id": 207, "name": "test-user", "uuid": "b32b5f06-a846-41f3-bd49-b485839386d3", "networkofferingid": 10} as a part of network shutdown
2025-12-12 08:49:54,456 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-2:[ctx-720b416c, job-119, ctx-d445e199]) (logid:82f88545) Network is not in the correct state to be destroyed: Implementing
2025-12-12 08:49:54,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Complete async job-119, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to delete network"}
2025-12-12 08:49:54,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Publish async job-119 complete on message bus
2025-12-12 08:49:54,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Wake up jobs related to job-119
2025-12-12 08:49:54,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Update db status for job-119
2025-12-12 08:49:54,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Wake up jobs joined with job-119 and disjoin all subjobs created from job- 119
2025-12-12 08:49:54,467 DEBUG [c.c.a.ApiServer] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Retrieved cmdEventType from job info: NETWORK.DELETE
2025-12-12 08:49:54,469 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Done executing org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd for job-119
2025-12-12 08:49:54,469 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-2:[ctx-720b416c, job-119]) (logid:82f88545) Remove job-119 from job monitoring
  1. Click on cleanup network
Image

Exception

Image

logs


root@acs420:~# cat   /var/log/cloudstack/management/management-server.log |grep -i "logid:b1f9f24f"
2025-12-12 08:57:08,455 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Executing AsyncJob {"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.network.RestartNetworkCmd","cmdInfo":"{\"cleanup\":\"true\",\"response\":\"json\",\"ctxUserId\":\"2\",\"sessionkey\":\"pbLHDerttlq72JFt8rB7UYkO_30\",\"httpmethod\":\"POST\",\"ctxStartEventId\":\"342\",\"id\":\"b32b5f06-a846-41f3-bd49-b485839386d3\",\"ctxDetails\":\"{\\\"interface com.cloud.network.Network\\\":\\\"b32b5f06-a846-41f3-bd49-b485839386d3\\\"}\",\"ctxAccountId\":\"2\",\"uuid\":\"b32b5f06-a846-41f3-bd49-b485839386d3\",\"cmdEventType\":\"NETWORK.RESTART\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":122,"initMsid":206863097656310,"instanceId":207,"instanceType":"Network","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"b1f9f24f-72af-4d6f-b268-88922545af6c"}
2025-12-12 08:57:08,460 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122, ctx-f7e4472a]) (logid:b1f9f24f) Account [Account [{"accountName":"admin","id":2,"uuid":"69c635a2-d5e9-11f0-a8f5-bc241173f3f6"}]] has access to resource.
2025-12-12 08:57:08,475 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Complete async job-122, jobStatus: FAILED, resultCode: 431, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"431","errortext":"Network is not in the right state to be restarted. Correct states are: Implemented, Setup"}
2025-12-12 08:57:08,476 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Publish async job-122 complete on message bus
2025-12-12 08:57:08,476 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Wake up jobs related to job-122
2025-12-12 08:57:08,476 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Update db status for job-122
2025-12-12 08:57:08,476 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Wake up jobs joined with job-122 and disjoin all subjobs created from job- 122
2025-12-12 08:57:08,481 DEBUG [c.c.a.ApiServer] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Retrieved cmdEventType from job info: NETWORK.RESTART
2025-12-12 08:57:08,482 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Done executing org.apache.cloudstack.api.command.user.network.RestartNetworkCmd for job-122
2025-12-12 08:57:08,482 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-5:[ctx-90bab3ae, job-122]) (logid:b1f9f24f) Remove job-122 from job monitoring

Currently, the only way is to update the database and then delete the network

What to do about it?

CloudStack should cleanup networks which are struck in implementing state

kiranchavala avatar Dec 12 '25 09:12 kiranchavala