seldon-core
seldon-core copied to clipboard
feat(grafana): add performance dashboard
New dashboard showing:
- Filter-able to a given set of models/inference server pods:
- per (model, inference server pod) throughput and average latency
- aggregated per model throughput
- aggregated per inference server pod throughput
- Filter-able to a given set of inference server pods
- latency heatmaps (configurable rate interval) . agent -> inference srv -> agent . inference srv -> model -> inference srv
- in-flight inference requests
- CPU usage
Which issue(s) this PR fixes: Fixes #INFRA-1141 (internal) - performance dashboard