google-cloud-go icon indicating copy to clipboard operation
google-cloud-go copied to clipboard

feat(bigtable): Client side merics

Open bhshkh opened this issue 9 months ago • 1 comments

This PR adds support for built-in client side metrics in Go Bigtable client. Metrics will be recorded and exported by default to GCM if not opted out by user.

Major changes in this PR:

  1. ClientConfig - New field MetricsProvider has been added. It can be used to opt out of metrics collection or provide user's own metric provider.
  2. metricsTracerFactory - Private field added on client. When a new client is created,
  • if user hasn't provided MetricsProvider option in config, a new meter provider is created with GCM exporter to record and export built-in metrics.
  • if user has provided NoopMetricsProvider in config, metrics will neither be recorded, nor be exported
  • if user has provided their own MeterProvider, it will used. During client creation, a new metricsTracerFactory instance will be created, then OTel (Open Telemetry) instruments will be created on meter provider and these instruments will be saved to metricsTracerFactory. Whenever any operation is invoked on data client such as ReadRows, a new builtinMetricsTracer will be created using the metricsTracerFactory
  1. bigtable/metric_monitoring_exporter.go file is the GCM (Google Cloud Monitoring) exporter. The code in this file is a modified version of https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/88c799af8373802d659e6efdca387e3a353e8c21/exporter/metric/metric.go :

    • Custom monitored resource type is being used here instead of deriving it from OpenTelemetry resource configuration
    • Metric attributes are being added to monitored resource labels while the original one adds OTel resource attributes to GCM monitored resource
    • Exporter exports only built in metrics to GCM
  2. All the data methods have been modified to include attempt recorder and operation recorder.

Out of all the metrics: https://cloud.google.com/bigtable/docs/client-side-metrics-descriptions, this PR adds operation_latencies, attempt_latencies, server_latencies and retry_count. Rest will be added in follow up PRs

Below diagram shows code flow when client is created using NewClient method: CBT client side metrics (2)

bhshkh avatar Apr 25 '24 22:04 bhshkh

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot https://conventionalcommits.org/