integrations-core icon indicating copy to clipboard operation
integrations-core copied to clipboard

Add Telemetry gatherer for database integrations

Open sethsamuel opened this issue 8 months ago • 1 comments

What does this PR do?

Adds a new Telemetry gatherer for database integrations. This component emits a debounced set of events once per minute to the dbm-metrics endpoint. See this PR for handling of the events.

Motivation

We want to gather (very coarse) cross-org metrics on our integration performance. These metrics are kept intentionally low in cardinality and frequency, but should be sufficient to flag problems with agents collecting too many rows or taking too long to gather information.

Additional Notes

These events are very small and emitted far less frequently than the database integration events they are monitoring. Performance impact should be immeasurably low, but there is a hidden config option (enable_telemetry) that can be set to false if there are unexpected problems.

Review checklist (to be filled by reviewers)

  • [ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • [ ] Changelog entries must be created for modifications to shipped code
  • [ ] Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • [ ] If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

sethsamuel avatar Jun 13 '24 15:06 sethsamuel