Fabio registers Consul Connect Proxy endpoint

[fredwangwang]

Hi, I am running Fabio as the http load balancer for the jobs deployed using Nomad. Recently I started to add consul connect (Service Mesh sidecar using Envoy) into the job file:

    service {
      name = "huan-service"
      port = "http"

      tags = ["http", "urlprefix-/aaaaaa/v1"]

      check {
        type     = "http"
        port     = "http"
        path     = "/"
        interval = "10s"
        timeout  = "60s"
      }

       // this is to enable consul connect envoy sidecar
      connect {
        sidecar_service {
          proxy {
            config {
              protocol = "http"
            }
          }
        }
      }
    }

The Envoy sidecar added as a result also expose a port on the host, but that port is secured by mTLS, so it is not accessable from outside the Service Mesh.

However, Fabio doesn't seem to differentiate between the port exposed by the job itself and the port exposed by Envoy sidecar, and it registers two entries in the routing table:

Because of that, 50% of the traffic just gets 502.

Although there is a workaround to register two different service sections with different names, 1 for fabio, 1 for consul connect; Is this something should be fixed in fabio to differentiate between normal service entry and consul connect service entry?

I could help to get a PR if you think its worth fixing!

Thank you

[tristanmorgan]

Yeah, because the sidecar and the service get the same tags. I was testing connect native services but because there isn't the separate service, I didn't hit the same issue. Related to #788 Fabio will need to understand what it can connect to.

[scorsi]

I'm facing the same issue. But I only got the sidecar-proxy routed. I did migrate from Traefik since they have the same issue... I hoped Fabio "built for Consul" should handle it... ^^

[MartinSchmidt123]

As a workaround, you can use set custom tags for the sidecar_service (see documentation), i.e.

sidecar_service {
    tags = ["http"]
    proxy {
        config {
            protocol = "http"
        }
    }
}

But keep in mind that the tag list has to contain at least one element (the default seems to be [] and in this case the group service tags are used.

[robustq]

Hm, I ran into this issue today on Fabio 1.15.5 , and like @scorsi , only the sidecar-proxy was routed in Fabio.

When I attempted to use @MartinSchmidt123 workaround, of adding tags to both the top-level service and the sidecar_service stanza, neither the sidecar-proxy nor the actual service were routed in Fabio, despite Consul showing the correct tags for the service (using the countdash example with Envoy Connect). Bummer!

[gbolo]

Hm, I ran into this issue today on Fabio 1.15.5 , and like @scorsi , only the sidecar-proxy was routed in Fabio.

When I attempted to use @MartinSchmidt123 workaround, of adding tags to both the top-level service and the sidecar_service stanza, neither the sidecar-proxy nor the actual service were routed in Fabio, despite Consul showing the correct tags for the service (using the countdash example with Envoy Connect). Bummer!

I ran into the same issue, however I resolved it by adding a healthcheck to the example countdash service. Fabio was then able to expose the app.

      check {
        name     = "dash-ping"
        type     = "http"
        path     = "/"
        interval = "5s"
        timeout  = "2s"
      }

As the previous comment suggested, you should also add a dummy tag to the sidecar_service so that fabio ignores it. otherwise it will get applied with the tags defined in the app service:

      connect {
        sidecar_service {
          tags = ["ignored-by-fabio"]
          proxy {
            upstreams {
              destination_name = "count-api"
              local_bind_port  = 8080
            }
          }
        }
      }