nerdctl icon indicating copy to clipboard operation
nerdctl copied to clipboard
verified publisher icon containerd

bridge network (?) appears to leave a bunch of dangling iptables rules

Open apostasie opened this issue 1 year ago • 9 comments

Description

Maybe this is related to #1872 ?

Steps to reproduce the issue

sudo nerdctl ps -a
CONTAINER ID    IMAGE    COMMAND    CREATED    STATUS    PORTS    NAMES


sudo iptables-save

-A POSTROUTING -s 10.4.0.197/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-1d73fd5bf60ed266779ac09831f72057e898c3fce6dd87002c9daff2ce3454a3\"" -j CNI-c4485ea363c66d916d2377b7
-A POSTROUTING -s 10.4.0.198/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ac90b9d0647a934baec840db19b3824d81ca7ca407b9bad271e47acf119c4c75\"" -j CNI-e0a50dffac3cd362832365a4
-A POSTROUTING -s 10.4.0.199/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-15bd207cf924c7a592bbe00abfec753f13130a938ce61cc06fb7a84e9105ad66\"" -j CNI-41eb0e14025647d3c24e405b
-A POSTROUTING -s 10.4.0.200/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-b30c753e86caf6b07426d0669b7718b9cd0548395ea5668e09ec691cd1d69233\"" -j CNI-7637801b9bcec2002c68b2bd
-A POSTROUTING -s 10.4.0.201/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-55309631e16733497b299211b23bbfffcdc531c0bd0ed68c344cae567fd9386b\"" -j CNI-408aef95f1ad1c819e61e379
-A POSTROUTING -s 10.4.0.202/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-00404f0bf146fa6ce2ead969b7618fb1563a117e8bd29550041881a4c985d1ab\"" -j CNI-58549c32ab4ba9b6138209da
-A POSTROUTING -s 10.4.0.203/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-4fd1a300ef4a22cddbfe4fedc95af8fe18da538bc16675d37cc22dde3578547a\"" -j CNI-e4412f3237604efd3c26bfc7
-A POSTROUTING -s 10.4.0.204/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-32e7c97a6c1d3a63ac3d1dd9ebd2651a4798064fc156439827148d6ea8632a14\"" -j CNI-e396c1c66ec726d98c17bbc2
-A POSTROUTING -s 10.4.0.205/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ca85f92c4762354ba06fe2a1a3cdf5f240b0641bf61513f501f6362027c2c617\"" -j CNI-2a8cc16295670ffc96de5a98
-A POSTROUTING -s 10.4.0.206/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9c62518f148e4bf79ab3f9fb3725864fad5b1be87afebecaaed150a6e940752e\"" -j CNI-98b8940dc9beddb2a80de3e2
-A POSTROUTING -s 10.4.0.207/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-49ab18dd64de04b5e2972757b99b89033badf5ae1919b24f897aaf52ad49eecf\"" -j CNI-53b720eecc3f3e2205d8fb4c
-A POSTROUTING -s 10.4.0.208/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-10ec46b10227bd113f57b1eddb92abeeb45f5960f9fe2f452a9b605afc4aea9e\"" -j CNI-e15eab9ee705e4b4e26e5e76
-A POSTROUTING -s 10.4.0.209/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c7c81fa4a302e3bdc3ad6be9a5302e9666c459524386f02d9f1bac136700dd8e\"" -j CNI-3a2406eb171bfdd258398c1f
-A POSTROUTING -s 10.4.0.210/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9617452bbd0ffa38209eac238f2ec642dbdd87647fd2b19cd84c38dcf3f40453\"" -j CNI-da7ea8f2c2ecff2b07594e55
-A POSTROUTING -s 10.4.0.211/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-47ec57db17ae0813a693757b4a4c1fc0b357f5100db37978a0960821fcc96287\"" -j CNI-a6a54776e68d8ba1ce02ab84
-A POSTROUTING -s 10.4.0.212/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c282fc8fc04cb31aef18d7277894a3a05f73db838272c3e58fda1b05f286c3f3\"" -j CNI-a9bbe4b91567faf57038efed
-A POSTROUTING -s 10.4.0.213/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-0118471dbae60dd72947924b88a43f80a8ff259c04b10f5ccaec4cf6964606fa\"" -j CNI-f2382284f9fd1fb616cec0fe
-A POSTROUTING -s 10.4.0.214/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-2a700024b4c55bbbe8e5b2e66e5d8ceb0f33178a791fa63c2e6a3c7cb1491ad5\"" -j CNI-ddd7bb888c5f08c080c4e18b
-A POSTROUTING -s 10.4.0.215/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-abb3d16d621abb1de10a6767e7e97ba517d380c415c4e0252db287674ee3790a\"" -j CNI-5d2a9ddcc32447b184c845aa
-A POSTROUTING -s 10.4.0.216/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-f2a846d8b3187a3301e96404b2b3ba067eb02213158e68173b422d8fc17df523\"" -j CNI-32225503aa1d2ef3ebb59115
-A POSTROUTING -s 10.4.0.217/32 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-696e8a6711cd4730503180386b144f7e7b78a87a972b567739a30c2eb2893f0c\"" -j CNI-1e3b8c304159d8c5064a34bb
-A POSTROUTING -s 10.4.0.218/32 -m comment --comment "name: \"bridge\" id: \"default-947747039cf87a4e4e3c92ca6a77b89c3643644a8dee335ded4ae4f336d71837\"" -j CNI-2e7f933c92f5d56b0936410d
-A CNI-1e3b8c304159d8c5064a34bb -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-696e8a6711cd4730503180386b144f7e7b78a87a972b567739a30c2eb2893f0c\"" -j ACCEPT
-A CNI-1e3b8c304159d8c5064a34bb ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-696e8a6711cd4730503180386b144f7e7b78a87a972b567739a30c2eb2893f0c\"" -j MASQUERADE
-A CNI-2a8cc16295670ffc96de5a98 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ca85f92c4762354ba06fe2a1a3cdf5f240b0641bf61513f501f6362027c2c617\"" -j ACCEPT
-A CNI-2a8cc16295670ffc96de5a98 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ca85f92c4762354ba06fe2a1a3cdf5f240b0641bf61513f501f6362027c2c617\"" -j MASQUERADE
-A CNI-2e7f933c92f5d56b0936410d -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"default-947747039cf87a4e4e3c92ca6a77b89c3643644a8dee335ded4ae4f336d71837\"" -j ACCEPT
-A CNI-2e7f933c92f5d56b0936410d ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"default-947747039cf87a4e4e3c92ca6a77b89c3643644a8dee335ded4ae4f336d71837\"" -j MASQUERADE
-A CNI-32225503aa1d2ef3ebb59115 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-f2a846d8b3187a3301e96404b2b3ba067eb02213158e68173b422d8fc17df523\"" -j ACCEPT
-A CNI-32225503aa1d2ef3ebb59115 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-f2a846d8b3187a3301e96404b2b3ba067eb02213158e68173b422d8fc17df523\"" -j MASQUERADE
-A CNI-3a2406eb171bfdd258398c1f -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c7c81fa4a302e3bdc3ad6be9a5302e9666c459524386f02d9f1bac136700dd8e\"" -j ACCEPT
-A CNI-3a2406eb171bfdd258398c1f ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c7c81fa4a302e3bdc3ad6be9a5302e9666c459524386f02d9f1bac136700dd8e\"" -j MASQUERADE
-A CNI-408aef95f1ad1c819e61e379 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-55309631e16733497b299211b23bbfffcdc531c0bd0ed68c344cae567fd9386b\"" -j ACCEPT
-A CNI-408aef95f1ad1c819e61e379 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-55309631e16733497b299211b23bbfffcdc531c0bd0ed68c344cae567fd9386b\"" -j MASQUERADE
-A CNI-41eb0e14025647d3c24e405b -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-15bd207cf924c7a592bbe00abfec753f13130a938ce61cc06fb7a84e9105ad66\"" -j ACCEPT
-A CNI-41eb0e14025647d3c24e405b ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-15bd207cf924c7a592bbe00abfec753f13130a938ce61cc06fb7a84e9105ad66\"" -j MASQUERADE
-A CNI-53b720eecc3f3e2205d8fb4c -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-49ab18dd64de04b5e2972757b99b89033badf5ae1919b24f897aaf52ad49eecf\"" -j ACCEPT
-A CNI-53b720eecc3f3e2205d8fb4c ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-49ab18dd64de04b5e2972757b99b89033badf5ae1919b24f897aaf52ad49eecf\"" -j MASQUERADE
-A CNI-58549c32ab4ba9b6138209da -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-00404f0bf146fa6ce2ead969b7618fb1563a117e8bd29550041881a4c985d1ab\"" -j ACCEPT
-A CNI-58549c32ab4ba9b6138209da ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-00404f0bf146fa6ce2ead969b7618fb1563a117e8bd29550041881a4c985d1ab\"" -j MASQUERADE
-A CNI-5d2a9ddcc32447b184c845aa -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-abb3d16d621abb1de10a6767e7e97ba517d380c415c4e0252db287674ee3790a\"" -j ACCEPT
-A CNI-5d2a9ddcc32447b184c845aa ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-abb3d16d621abb1de10a6767e7e97ba517d380c415c4e0252db287674ee3790a\"" -j MASQUERADE
-A CNI-7637801b9bcec2002c68b2bd -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-b30c753e86caf6b07426d0669b7718b9cd0548395ea5668e09ec691cd1d69233\"" -j ACCEPT
-A CNI-7637801b9bcec2002c68b2bd ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-b30c753e86caf6b07426d0669b7718b9cd0548395ea5668e09ec691cd1d69233\"" -j MASQUERADE
-A CNI-98b8940dc9beddb2a80de3e2 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9c62518f148e4bf79ab3f9fb3725864fad5b1be87afebecaaed150a6e940752e\"" -j ACCEPT
-A CNI-98b8940dc9beddb2a80de3e2 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9c62518f148e4bf79ab3f9fb3725864fad5b1be87afebecaaed150a6e940752e\"" -j MASQUERADE
-A CNI-a6a54776e68d8ba1ce02ab84 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-47ec57db17ae0813a693757b4a4c1fc0b357f5100db37978a0960821fcc96287\"" -j ACCEPT
-A CNI-a6a54776e68d8ba1ce02ab84 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-47ec57db17ae0813a693757b4a4c1fc0b357f5100db37978a0960821fcc96287\"" -j MASQUERADE
-A CNI-a9bbe4b91567faf57038efed -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c282fc8fc04cb31aef18d7277894a3a05f73db838272c3e58fda1b05f286c3f3\"" -j ACCEPT
-A CNI-a9bbe4b91567faf57038efed ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-c282fc8fc04cb31aef18d7277894a3a05f73db838272c3e58fda1b05f286c3f3\"" -j MASQUERADE
-A CNI-c4485ea363c66d916d2377b7 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-1d73fd5bf60ed266779ac09831f72057e898c3fce6dd87002c9daff2ce3454a3\"" -j ACCEPT
-A CNI-c4485ea363c66d916d2377b7 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-1d73fd5bf60ed266779ac09831f72057e898c3fce6dd87002c9daff2ce3454a3\"" -j MASQUERADE
-A CNI-da7ea8f2c2ecff2b07594e55 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9617452bbd0ffa38209eac238f2ec642dbdd87647fd2b19cd84c38dcf3f40453\"" -j ACCEPT
-A CNI-da7ea8f2c2ecff2b07594e55 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-9617452bbd0ffa38209eac238f2ec642dbdd87647fd2b19cd84c38dcf3f40453\"" -j MASQUERADE
-A CNI-ddd7bb888c5f08c080c4e18b -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-2a700024b4c55bbbe8e5b2e66e5d8ceb0f33178a791fa63c2e6a3c7cb1491ad5\"" -j ACCEPT
-A CNI-ddd7bb888c5f08c080c4e18b ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-2a700024b4c55bbbe8e5b2e66e5d8ceb0f33178a791fa63c2e6a3c7cb1491ad5\"" -j MASQUERADE
-A CNI-e0a50dffac3cd362832365a4 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ac90b9d0647a934baec840db19b3824d81ca7ca407b9bad271e47acf119c4c75\"" -j ACCEPT
-A CNI-e0a50dffac3cd362832365a4 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-ac90b9d0647a934baec840db19b3824d81ca7ca407b9bad271e47acf119c4c75\"" -j MASQUERADE
-A CNI-e15eab9ee705e4b4e26e5e76 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-10ec46b10227bd113f57b1eddb92abeeb45f5960f9fe2f452a9b605afc4aea9e\"" -j ACCEPT
-A CNI-e15eab9ee705e4b4e26e5e76 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-10ec46b10227bd113f57b1eddb92abeeb45f5960f9fe2f452a9b605afc4aea9e\"" -j MASQUERADE
-A CNI-e396c1c66ec726d98c17bbc2 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-32e7c97a6c1d3a63ac3d1dd9ebd2651a4798064fc156439827148d6ea8632a14\"" -j ACCEPT
-A CNI-e396c1c66ec726d98c17bbc2 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-32e7c97a6c1d3a63ac3d1dd9ebd2651a4798064fc156439827148d6ea8632a14\"" -j MASQUERADE
-A CNI-e4412f3237604efd3c26bfc7 -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-4fd1a300ef4a22cddbfe4fedc95af8fe18da538bc16675d37cc22dde3578547a\"" -j ACCEPT
-A CNI-e4412f3237604efd3c26bfc7 ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-4fd1a300ef4a22cddbfe4fedc95af8fe18da538bc16675d37cc22dde3578547a\"" -j MASQUERADE
-A CNI-f2382284f9fd1fb616cec0fe -d 10.4.0.0/24 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-0118471dbae60dd72947924b88a43f80a8ff259c04b10f5ccaec4cf6964606fa\"" -j ACCEPT
-A CNI-f2382284f9fd1fb616cec0fe ! -d 224.0.0.0/4 -m comment --comment "name: \"bridge\" id: \"nerdctl-test-0118471dbae60dd72947924b88a43f80a8ff259c04b10f5ccaec4cf6964606fa\"" -j MASQUERADE

Describe the results you received and expected

na

What version of nerdctl are you using?

main

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

No response

apostasie avatar Jul 28 '24 16:07 apostasie

Wondering if CNI plugins project is still actively maintained?

Issue and PR got closed by a bot without any maintainers' feedback whatsoever:

  • https://github.com/containernetworking/plugins/pull/811
  • https://github.com/containernetworking/plugins/issues/810

apostasie avatar Jul 28 '24 16:07 apostasie

Pinged the maintainers:

  • https://github.com/containernetworking/plugins/pull/811#issuecomment-2259042708

AkihiroSuda avatar Jul 30 '24 19:07 AkihiroSuda

Crickets.

I really hate the idea of maintaining a (friendly) fork over here, but it might be time to consider it.

apostasie avatar Aug 21 '24 22:08 apostasie

Opened an issue about the health of the project:

  • https://github.com/containernetworking/plugins/issues/1077

AkihiroSuda avatar Aug 22 '24 03:08 AkihiroSuda

Likely the same as #3488

Maybe #3487 as well.

Unless I am missing something, it feels like one just cannot use nerdctl with containers exposing a port over bridge being started / destroyed a few times.

This would also explain a lot of flakyness on the CI where we successfully start containers exposing a port, but fail to connect to them (definitely affects ipfs, compose and registry tests).

I am now running with the patch from qkboy, which does seem to improve the situation.

I would make this a blocker for v2.......

apostasie avatar Oct 02 '24 23:10 apostasie

Upstream is still pending...

apostasie avatar Apr 21 '25 15:04 apostasie

@AkihiroSuda why closing ticket https://github.com/containernetworking/plugins/issues/1077 ?

Should we use an OCI post-hook to clean up resources related to CNI for now?

fahedouch avatar May 12 '25 18:05 fahedouch

@AkihiroSuda why closing ticket containernetworking/plugins#1077 ?

Because the repo seemed to regain activities

Should we use an OCI post-hook to clean up resources related to CNI for now?

SGTM

AkihiroSuda avatar May 13 '25 03:05 AkihiroSuda

workaround, I will keep this ticket open to follow the upstream fix

fahedouch avatar Jun 06 '25 15:06 fahedouch