Response packets matched by wrong router

I have some router4s defined and they are working for "request" traffic. But the "response" traffic gets caught up in the wrong router in some cases.

This is a simplified re-creation of my scenario:

router4 FROM4_LAN0 inface "eth0" src "192.168.0.0/24"
	group with dst "192.168.0.0/16"
		route all reject with icmp-admin-prohibited dst not "192.168.1.0/24,192.168.7.0/24,192.168.9.0/24"
	group end
	route all accept

router4 FROM4_LAN1 inface "eth1" src "192.168.1.0/24"
	group with dst "192.168.0.0/16"
		route all reject with icmp-admin-prohibited dst not "192.168.7.0/24,192.168.9.0/24"
	group end
	route all accept

router4 FROM4_LAN2 inface "eth2" src "192.168.2.0/24"
	group with dst "192.168.0.0/16"
		route all reject with icmp-admin-prohibited dst not "192.168.1.0/24,192.168.7.0/24,192.168.9.0/24"
	group end
	route all accept

Packet capture in this situation tells me the following:

• Clients in LAN0 can make requests to a web server in LAN1. The server in LAN1 can respond to the requests. This is the desired behaviour.
• Clients in LAN2 can also make requests to the server in LAN1. But when the server in LAN1 tries to respond, eth1 sends the server an ICMP-admin-prohibited and the client's request times out. This is undesired behaviour.

In this simplified example, I could move LAN1 to the bottom of the config and problem would be solved. But in my actual real configuration, I cannot just swap the order of routers that easily, because fixing the order of one thing will break the order of something else.

Is this a bug? Or expected behaviour and I need to figure out a different way of defining and ordering my routers?

Bump. Is this a bug? I feel like this behaviour goes against Firehol's usual "you don't have to do anything about replies" paradigm. Please can anyone confirm if what I'm seeing is expected?

I don't think this is a bug.

I haven't had time to analyse this exhaustively, however: your routers have all been set up with a source only. To detect the reverse paths, all firehol does is generate rules with switched src and dst (and sport and dport).

The iptables rules themselves are processed in order, so I think it will be possible to create scenarios such as you describe.

Looking at what you have done, you might consider creating a set of routers with a "dst not" consisting of the one you currently have in the "group" for each, and make the whole router do nothing but reject.

Below these you can have one or more more general routers which accept. These will only ever be evaluted if one of the prior routers has not matched.

In general you need to take care wherever you have some routers that are supersets of others; you need to make sure the more specific routers are processed first, as the rules are handled in order. If you have overlapping ranges, you probably need to set up a specific router for the overlap and handle the disjoint parts further down the rules.

Hope that helps...