whalewall icon indicating copy to clipboard operation
whalewall copied to clipboard

Unable to create rules when container not started

Open rgomezceis opened this issue 1 year ago • 26 comments

Using this rule:

- network: default
  container: test_container

If the container 'test_container' isn't running whalewall can't create all the rules for this service. That's correct but what happen when you have multiple stacks (docker compose files) and do a server reboot. Whalewall won't create the rules for some services because some containers isn't running (all the stacks are deployed at the same time)

So we can control the start order for services on the same docker compose file with 'depends_on' label, but not the services in other compose files. We could control this with a health check instead.

Is possible to create the rules only when the container is in healthy state? We can do a health check to check if the dependant container has already started and then create the whalewall rules when the service has the healthy label. This can be configurable with a label like 'whalewall.healthy: true' if true create the rules when container is in healthy state, and if isn't set create the rules when container starts.

Or do 3-5 retries until the container starts?

I don't know how to solve this problem, because if i use static ip for containers i need to set rules in both containers.

  • loki container: 172.27.0.2
  • promtail container: 172.27.0.3
  • Only outgoing traffic from promtail to loki:3100

Loki:

whalewall.enabled: true

Promtail:

      whalewall.enabled: true
      whalewall.rules: |
        output:
          # push to loki
          - ip: "172.27.0.2"
            proto: tcp
            port: 3100

Dec 27 11:17:11 xxxxxxx kernel: [ 5433.189077] whalewall-loki-16c68c683925 drop: IN=xxxxxxxx OUT=xxxxxxxx PHYSIN=xxxxxxx PHYSOUT=xxxxxxxx MAC=xxxxxxxxxxxxxx SRC=172.27.0.2 DST=172.27.0.3 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=3100 DPT=47836 WINDOW=65160 RES=0x00 ACK SYN URGP=0

nft chain:

        chain whalewall-loki-16c68c683925 {
                counter packets 630 bytes 37800 log prefix "whalewall-loki-16c68c683925 drop: " level info drop
        }

        chain whalewall-promtail-4f3d6d98bee7 {
                ip saddr 172.27.0.3 ip daddr 172.27.0.2 tcp dport 3100 ct state established,related,new counter packets 32 bytes 1920 accept
                ip saddr 172.27.0.2 ip daddr 172.27.0.3 tcp sport 3100 ct state established,related counter packets 0 bytes 0 accept 
                counter packets 33 bytes 1980 log prefix "whalewall-promtail-4f3d6d98bee7 drop: " level info drop
        }

Rule "ip saddr 172.27.0.2 ip daddr 172.27.0.3 tcp sport 3100 ct state established,related counter packets 0 bytes 0 accept" not working because is blocking by "log prefix "whalewall-loki-16c68c683925 drop:" first.

Traffic will be blocked in the opposite side (idk the reason of this traffic because it's only promtail -> loki). But when using container label, everything works:

      whalewall.enabled: true
      whalewall.rules: |
        output:
          # push to loki
          - network: logging_network
            container: loki
            proto: tcp
            port: 3100

nft chain:

    chain whalewall-loki-16c68c683925 {
            ip saddr 172.27.0.2 ip daddr 172.27.0.3 tcp sport 3100 ct state established,related counter packets 0 bytes 0 accept
            counter packets 698 bytes 41880 log prefix "whalewall-loki-16c68c683925 drop: " level info drop
    }

    chain whalewall-promtail-24f8a974fb5d {
            ip saddr 172.27.0.3 ip daddr 172.27.0.2 tcp dport 3100 ct state established,related,new counter packets 0 bytes 0 accept
            counter packets 3 bytes 180 log prefix "whalewall-promtail-24f8a974fb5d drop: " level info drop
    }

rgomezceis avatar Dec 27 '22 11:12 rgomezceis