smi-spec icon indicating copy to clipboard operation
smi-spec copied to clipboard

traffic metrics check in

Open michelleN opened this issue 4 years ago • 13 comments

At this point, there are both linkerd and istio implementations of the traffic metrics piece of the spec in the smi-metric repo.

@grampelberg @pothulapati - do ya'll have any thoughts or want to propose any changes at this point? Would love to hear feedback from any other folks who have implemented it as well.

michelleN avatar Sep 25 '19 01:09 michelleN

I have been working with @nicholasjackson to add the support for Consul Connect! It required k8s metadata to be a part of the envoy metrics. Once that is done, It wouldnt be that hard to add the support for it.

Also, @nicholasjackson has some idea's on improving the spec.

Pothulapati avatar Sep 25 '19 06:09 Pothulapati

I'd love to have a conversation with the Kiali folks to get a list of everything that's missing and what we can do to help out there.

The biggest metrics iteration I'd like to see is hooking up the ability to get a little bit more historical data: query by window, query by date, return a time series.

grampelberg avatar Sep 25 '19 14:09 grampelberg

@grampelberg do u know the Kiali folks' github handles? How do we get in touch with them?

michelleN avatar Sep 25 '19 21:09 michelleN

Alas, I don't have any good connections, just a ton of respect for the product.

grampelberg avatar Sep 25 '19 21:09 grampelberg

cc @lucasponce @jotak @jmazzitelli

simonpasquier avatar Sep 26 '19 07:09 simonpasquier

Hi @michelleN @grampelberg , We discussed a little bit about that a couple of months ago ( https://github.com/deislabs/smi-spec/issues/43 ) but TBH there hasn't been much time spent here on our side. I don't know if there's some news in SMI spec about the items discussed in that issue?

jotak avatar Sep 26 '19 07:09 jotak

Hi @michelleN we have IRC in freenode or Slack in istio.io. Also you can create any issue in Kiali project as well.

lucasponce avatar Sep 26 '19 07:09 lucasponce

@jotak that's a great starting point, I wanted to make sure that was all still valid and such. I'm gonna need to dig into the kiali code a little bit to get a better feel for how you're using everything.

grampelberg avatar Sep 26 '19 15:09 grampelberg

The core kiali graph code is here: https://github.com/kiali/kiali/blob/master/graph/telemetry/istio/istio.go

You can see how we utilize the telemetry in Prometheus.

@jshaughn is the subject-matter-expert for that code

jmazzitelli avatar Sep 26 '19 15:09 jmazzitelli

Alas, I don't have any good connections, just a ton of respect for the product.

@grampelberg thanks for the feedback! Happy to see this.

abonas avatar Sep 26 '19 19:09 abonas

@grampelberg A little while back I re-organized the graph code in Kiali to better prepare for possible addition of a non-istio telemetry vendor. If SMI now offers a fleshed-out API that can front-end Istio it would be a nice experiment to see what we could generate using the SMI telemetry API in addition to having the explicit Istio impl. If promising then we'd be able to potentially provide some support for non-istio mesh, like Linkerd and whatever else that may become available.

jshaughn avatar Sep 26 '19 19:09 jshaughn

@jshaughn We now have a functioning implementation of SMI metrics for Istio here. How can we help with getting Kiali to attempt to utilize the SMI metrics APIs?

lachie83 avatar Oct 02 '19 18:10 lachie83

I guess what would need to happen is to start work on a Kiali graph "telemetry vendor". Today the only supported vendor is "istio", meaning we directly query prometheus for istio metrics, and we directly query istio/k8s for any additional resource info (namespaces, services, worklods, etc). The root of the code is here. The api package shows the entry points to graph generation. In telemetry/istio is the istio package, holding all of the graph gen code. We'd want to start a new provider like telemetry\smi that could start serving up a valid TrafficMap data structure in response to API requests. It won't be super straightforward but perhaps you could use SMI "deployment" resources for "workload" or something like that. Kiali does not try to visualize pod-level metrics, but rather higher level aggregates.

jshaughn avatar Oct 07 '19 14:10 jshaughn