checkpoint-restore
iptables: Bad rule (does a matching rule exist in that chain?)
The socket-tcp-syn-sent test, running with the ns/uns flavors, fails in CircleCI when CRIU is built with libnftables-dev. Investigating further, it appears that CRIU does not restore the iptables rules created by the test. When this package is removed and CRIU is recompiled, it works as expected.
I was not able to reproduce this problem with newer kernel on Fedora.
$ uname -a
Linux ip-10-0-53-248 6.8.0-1019-aws #21~22.04.1-Ubuntu SMP Thu Nov 7 17:33:30 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
$ iptables --version
iptables v1.8.7 (nf_tables)
$ git diff
diff --git a/test/zdtm/static/socket-tcp-syn-sent.c b/test/zdtm/static/socket-tcp-syn-sent.c
index 8265d5fb7..76917240b 100644
--- a/test/zdtm/static/socket-tcp-syn-sent.c
+++ b/test/zdtm/static/socket-tcp-syn-sent.c
@@ -75,6 +75,8 @@ int main(int argc, char **argv)
if (system(cmd))
return -1;
+ system("iptables -L INPUT -n -v --line-numbers");
+
/* Construct the server address structure */
memset(&addr, 0, sizeof(addr));
if (ZDTM_FAMILY == AF_INET) {
@@ -96,6 +98,8 @@ int main(int argc, char **argv)
test_daemon();
test_waitsig();
+ system("iptables -L INPUT -n -v --line-numbers");
+
snprintf(cmd, sizeof(cmd), "iptables -w -t filter --protocol tcp -D INPUT --dport %d -j DROP", port);
if (system(cmd))
return -1;
The test output below shows that the DROP rule has not been restored:
python3 ./zdtm.py run -t zdtm/static/socket-tcp-syn-sent -f ns
userns is supported
Checking feature tcp_half_closed
tcp_half_closed is supported
Checking feature has_ipt_legacy
has_ipt_legacy is supported
=== Run 1/1 ================ zdtm/static/socket-tcp-syn-sent
================== Run zdtm/static/socket-tcp-syn-sent in ns ===================
DEP socket-tcp-syn-sent.d
CC socket-tcp-syn-sent.o
LINK socket-tcp-syn-sent
Construct root for zdtm/static/socket-tcp-syn-sent
Start test
Test is SUID
Running zdtm/static/socket-tcp-syn-sent.hook(--post-start)
./socket-tcp-syn-sent --pidfile=socket-tcp-syn-sent.pid --outfile=socket-tcp-syn-sent.out
Running zdtm/static/socket-tcp-syn-sent.hook(--pre-dump)
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 1 0.0.0.0:8880 0.0.0.0:*
SYN-SENT 0 1 127.0.0.1:51469 127.0.0.1:8880
Run criu dump
Running zdtm/static/socket-tcp-syn-sent.hook(--pre-restore)
Run criu restore
=[log]=> dump/zdtm/static/socket-tcp-syn-sent/135/1/restore.log
------------------------ grep Error ------------------------
b'(00.003191) 1: No ipcns-sem-11.img image'
b'(00.003835) 1: net: Try to restore a link 10:1:lo'
b'(00.003837) 1: net: Restoring link lo type 1'
b'(00.004231) 1: net: \tRunning ip addr restore'
b'Error: ipv4: Address already assigned.'
b'Error: ipv6: address already assigned.'
------------------------ ERROR OVER ------------------------
Running zdtm/static/socket-tcp-syn-sent.hook(--post-restore)
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 1 0.0.0.0:8880 0.0.0.0:*
SYN-SENT 0 1 127.0.0.1:51469 127.0.0.1:8880
Check TCP images
Send the 15 signal to 172
Wait for zdtm/static/socket-tcp-syn-sent(172) to die for 0.100000
########## Test zdtm/static/socket-tcp-syn-sent FAIL at result check ###########
Test output: ================================
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8880
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 1 60 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0
iptables: Bad rule (does a matching rule exist in that chain?).
<<< ================================
Running zdtm/static/socket-tcp-syn-sent.hook(--clean)
##################################### FAIL #####################################
When removing this package and rebuilding CRIU, the test passes:
python3 ./zdtm.py run -t zdtm/static/socket-tcp-syn-sent -f ns
userns is supported
Checking feature tcp_half_closed
tcp_half_closed is supported
Checking feature has_ipt_legacy
has_ipt_legacy is supported
=== Run 1/1 ================ zdtm/static/socket-tcp-syn-sent
================== Run zdtm/static/socket-tcp-syn-sent in ns ===================
DEP socket-tcp-syn-sent.d
CC socket-tcp-syn-sent.o
LINK socket-tcp-syn-sent
Construct root for zdtm/static/socket-tcp-syn-sent
Start test
Test is SUID
Running zdtm/static/socket-tcp-syn-sent.hook(--post-start)
./socket-tcp-syn-sent --pidfile=socket-tcp-syn-sent.pid --outfile=socket-tcp-syn-sent.out
Running zdtm/static/socket-tcp-syn-sent.hook(--pre-dump)
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 1 0.0.0.0:8880 0.0.0.0:*
SYN-SENT 0 1 127.0.0.1:48163 127.0.0.1:8880
Run criu dump
Running zdtm/static/socket-tcp-syn-sent.hook(--pre-restore)
Run criu restore
=[log]=> dump/zdtm/static/socket-tcp-syn-sent/135/1/restore.log
------------------------ grep Error ------------------------
b'(00.002958) 1: No ipcns-sem-11.img image'
b'(00.003607) 1: net: Try to restore a link 10:1:lo'
b'(00.003610) 1: net: Restoring link lo type 1'
b'(00.003964) 1: net: \tRunning ip addr restore'
b'Error: ipv4: Address already assigned.'
b'Error: ipv6: address already assigned.'
------------------------ ERROR OVER ------------------------
Running zdtm/static/socket-tcp-syn-sent.hook(--post-restore)
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 1 0.0.0.0:8880 0.0.0.0:*
SYN-SENT 0 1 127.0.0.1:48163 127.0.0.1:8880
Check TCP images
Send the 15 signal to 168
Wait for zdtm/static/socket-tcp-syn-sent(168) to die for 0.100000
Wait for zdtm/static/socket-tcp-syn-sent(168) to die for 0.200000
Wait for zdtm/static/socket-tcp-syn-sent(168) to die for 0.400000
Wait for zdtm/static/socket-tcp-syn-sent(168) to die for 0.800000
Running zdtm/static/socket-tcp-syn-sent.hook(--clean)
Removing dump/zdtm/static/socket-tcp-syn-sent/135
================== Test zdtm/static/socket-tcp-syn-sent PASS ===================
The test output below shows that the DROP rule exists after restore:
$ cat zdtm/static/socket-tcp-syn-sent.out
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8880
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8880
17:30:29.750: 5: PASS
A friendly reminder that this issue had no activity for 30 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
I encountered this failure in zdtm/static/socket-tcp-syn-sent during the cleanup phase on Ubuntu 22.04 with AWS Kernel 6.8. It appears to be related to the known iptables-nft v1.8.7 rule canonicalization issue, where iptables fails to find the matching rule to delete, returning Bad rule.
I can reliably fix this by changing the specific deletion command (-D) to a chain flush:
snprintf(cmd, sizeof(cmd), "iptables -w -t filter -F INPUT");
My question is: Is it safe to flush the entire INPUT chain in this test context? Does the ZDTM harness or the namespace initialization populate the INPUT chain with any other essential rules that would be accidentally removed by a flush, or is the chain guaranteed to be owned solely by this test case?
Here is my machine configuration:
ubuntu@ip-172-31-92-244:~/criu$ uname -a
Linux ip-172-31-92-244 6.8.0-1040-aws #42~22.04.1-Ubuntu SMP Wed Sep 24 10:26:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ip-172-31-92-244:~/criu$ iptables --version
iptables v1.8.7 (nf_tables)
And I think the file socket-tcp-closed.c has same problem which will cause test fail. Maybe we can use the command(-D) to fix this bug:
snprintf(cmd, sizeof(cmd), "iptables -w -t filter -F INPUT");
I'd appreciate it if you can give me any help :)