spanner: Native Metrics Enabled by Default in v1.71.0+ Causes Permission Errors

[hodge1997]

Description After upgrading to Spanner client library versions >v1.71.0, our application began failing with permission errors (monitoring.timeSeries.create denied) in production. This is caused by the new native metrics collection being enabled by default, which requires the monitoring.metricWriter role. While disabling via DisableNativeMetrics: true resolves the issue, we believe this behavior introduces an unexpected breaking change and poses risks for existing users.

// Current behavior (v1.71.0+) causes errors unless explicitly disabled
client, err := spanner.NewClientWithConfig(ctx, database, spanner.ClientConfig{
    DisableNativeMetrics: true, // Required to avoid permission issues
})

Proposed Fixes Change Default to DisableNativeMetrics: true: Revert to disabling metrics by default to avoid breaking existing deployments.

Environment Library Version: v1.71.0+

GCP Environment: Cloud Run / GKE / etc.

Permissions: Service account lacks monitoring.metricWriter.

[rahul2393]

Hello @hodge1997 Exporter runs in background so it should not break the production code, worst case scenario is exporter will log permission denied error. Can you share stack trace and what exactly you see

[ensonic]

We're experiencing the same issue. We're getting a lot of these permission denied errors logged (once every second). The service code that uses the spanner client libraries uses workload identity and the mapped GCP IAM service account has roles/spanner.databaseUser and hence monitoring.timeSeries.create. @rahul2393 - what do you mean by "Exporter runs in the background", is it a go-routine in the same proc? Any idea what it is triggering the errors, despite the IAM setup being sufficient for the actual workload.

[hodge1997]

I have to set the environment variables to fix it.

SPANNER_DISABLE_BUILTIN_METRICS true