testkube icon indicating copy to clipboard operation
testkube copied to clipboard

Not all metrics are available.

Open windowsrefund opened this issue 9 months ago • 8 comments

Using Grafana's explorer, I do not see the following metrics:

  • testkube_testtriggers_creations_count
  • testkube_testtriggers_updates_count
  • testkube_testtriggers_deletes_count
  • testkube_testtriggers_bulk_updates_count
  • testkube_testtriggers_bulk_deletes_count

To Reproduce Steps to reproduce the behavior:

  1. Ensure the testkube-api-server-servicemonitor ServiceMonitor is configured.
  2. Verify other metrics are seen in Grafana Metrics Explorer (testkube_test*).
  3. Attempt to query for metrics starting with testkube_testtr

Expected behavior

All metrics mentioned in the documentation (linked above) are published.

Version / Cluster

  • Testkube 1.17.21

windowsrefund avatar May 13 '24 15:05 windowsrefund

thank you @windowsrefund will check

vsukhin avatar May 13 '24 15:05 vsukhin

so, these metrics are used when you create/update/dellete test triggers using api, did you call it?

vsukhin avatar May 14 '24 07:05 vsukhin

I've created triggers via kubectl apply ... Are you saying these metrics are only published if the triggers are created using the testkube binary or dashboard?

windowsrefund avatar May 14 '24 14:05 windowsrefund

think so, @dejanzele ?

vsukhin avatar May 14 '24 15:05 vsukhin

If so, that's quite the limited use case as I'd tend to think larger shops aren't going to be working in that capacity to create and deploy their triggers. Instead, you're going to fold those bits into a Helm deployment (or something) when you're deploying across more than a few clusters.

windowsrefund avatar May 14 '24 15:05 windowsrefund

agrree, need to change

vsukhin avatar May 14 '24 15:05 vsukhin

hey, @windowsrefund I rechecked the code and it sound like a more global thing. We do collect information when test or testsuite crd is created in the cluster and send it for telemtry purposes, if it's enabled. But prometheus metrics are linked to the particular testkube api server instance and reflect activity in this instance and you can have multiple testkube api serverr instances in general. So if crd is created somewhere in Argo flows, we can't simply increase promethues metrics for the first available tekube api server instance

might be, we need to send them not only to telemetry server, but to the metrcis server as well

vsukhin avatar May 16 '24 11:05 vsukhin

might be we can add such metrics to operator, because itt's responsible for recomcilliation of all Testkueb CRD

vsukhin avatar May 16 '24 16:05 vsukhin