seldon-core icon indicating copy to clipboard operation
seldon-core copied to clipboard

feat(grafana): add performance dashboard

Open lc525 opened this issue 1 year ago • 0 comments

New dashboard showing:

  • Filter-able to a given set of models/inference server pods:
    • per (model, inference server pod) throughput and average latency
    • aggregated per model throughput
    • aggregated per inference server pod throughput
  • Filter-able to a given set of inference server pods
    • latency heatmaps (configurable rate interval) . agent -> inference srv -> agent . inference srv -> model -> inference srv
    • in-flight inference requests
    • CPU usage

Which issue(s) this PR fixes: Fixes #INFRA-1141 (internal) - performance dashboard

lc525 avatar Aug 20 '24 16:08 lc525