old_issues_repo icon indicating copy to clipboard operation
old_issues_repo copied to clipboard

Filter health probing checks from Tracing

Open day0ops opened this issue 7 years ago • 5 comments
trafficstars

Is there anyway to filter out K8 health check probes from Tracing ?

I have been looking at few old posts about this question but none of give any solution. Everything is pointing to adding this as a feature to Envoy.

day0ops avatar May 10 '18 14:05 day0ops

We are also looking for same feature from istio proxy

https://stackoverflow.com/questions/50594137/how-can-i-control-which-data-gets-reported-to-the-istio-mixer-by-the-side-car

dgurindapalli avatar May 29 '18 23:05 dgurindapalli

It is possible in envoy to configure the health check endpoint, and then it will be ignored by tracing. For example, I updated the jaeger tracing example front-envoy-jaeger.yaml to include the health check filter:

          http_filters:
          - name: envoy.health_check
            config:
              endpoint: "/health"
              pass_through_mode: false
          - name: envoy.router
            config: {}

I then called the app using curl -v http://localhost:8000/trace/1 and saw a trace as expected in the Jaeger UI. Then I issued a request on the health endpoint:

$ curl -v http://localhost:8000/health
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
> GET /health HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.59.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< x-envoy-upstream-healthchecked-cluster: front-proxy
< date: Wed, 30 May 2018 09:43:40 GMT
< server: envoy
< content-length: 0
< 
* Connection #0 to host localhost left intact

which did not result in a trace instance.

The issue is I currently don't know how this v2 config can be passed from Istio to the Envoy proxy. Hopefully someone with knowledge of the v2 api can answer that.

objectiser avatar May 30 '18 09:05 objectiser

@nixgadget Just wondered, have you tried adding the http header X-B3-Sampled: 0 to the liveness probe?

objectiser avatar Jun 02 '18 14:06 objectiser

Honestly didnt know about that header. Il look into it and report back

day0ops avatar Jun 02 '18 15:06 day0ops

Have tested it, and it works, e.g.

        livenessProbe:
          httpGet:
            path: "/health"
            port: 8080
            httpHeaders:
            - name: X-B3-Sampled
              value: "0"
          initialDelaySeconds: 60
          periodSeconds: 10

objectiser avatar Jun 06 '18 15:06 objectiser