stackdriver_exporter icon indicating copy to clipboard operation
stackdriver_exporter copied to clipboard

Internal server error for custom metrics

Open lunaqdm opened this issue 7 years ago • 1 comments

I have some user-defined metrics in stackdriver (in particular, for dataflow) that I am having trouble exporting. When I set the environment variable STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES to, for example bigtable.googleapis.com/cluster, everything works fine and I am able to see my metrics in Prometheus. However, when I instead set it to bigtable.googleapis.com/cluster,custom.googleapis.com, I get 500s when I attempt to get metrics.

HTTP request sent, awaiting response... 500 Internal Server Error
2018-10-03 18:59:19 ERROR 500: Internal Server Error.

Has anyone been able to successfully export custom metrics, and if so how?

lunaqdm avatar Oct 03 '18 19:10 lunaqdm

I see the same issue with 0.6.0. There does not seem to be a response body, just a 500.

Metrics
STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES:  pubsub.googleapis.com/subscription/ack_message_count,pubsub.googleapis.com/subscription/num_outstanding_messages,pubsub.googleapis.com/subscription/num_undelivered_messages,pubsub.googleapis.com/subscription/oldest_unacked_message_age,pubsub.googleapis.com/topic/send_request_count,cloudsql.googleapis.com/database/cpu/usage_time,cloudsql.googleapis.com/database/cpu/utilization,cloudsql.googleapis.com/database/disk/bytes_used,cloudsql.googleapis.com/database/disk/quota,cloudsql.googleapis.com/database/disk/read_ops_count,cloudsql.googleapis.com/database/disk/write_ops_count,cloudsql.googleapis.com/database/disk/utilization,cloudsql.googleapis.com/database/memory/quota,cloudsql.googleapis.com/database/memory/usage,cloudsql.googleapis.com/database/memory/utilization,cloudsql.googleapis.com/database/mysql/innodb_data_fsyncs,cloudsql.googleapis.com/database/mysql/queries,cloudsql.googleapis.com/database/mysql/questions,cloudsql.googleapis.com/database/mysql/received_bytes_count,cloudsql.googleapis.com/database/mysql/sent_bytes_count,cloudsql.googleapis.com/database/network/connections,cloudsql.googleapis.com/database/network/received_bytes_count,cloudsql.googleapis.com/database/network/sent_bytes_count,cloudsql.googleapis.com/database/uptime,redis.googleapis.com/clients/connected,redis.googleapis.com/commands/calls,redis.googleapis.com/commands/total_time,redis.googleapis.com/keyspace/keys,redis.googleapis.com/keyspace/keys_with_expiration,redis.googleapis.com/replication/master/slaves/lag,redis.googleapis.com/stats/cache_hit_ratio,redis.googleapis.com/stats/connections/total,redis.googleapis.com/stats/cpu_utilization,redis.googleapis.com/stats/evicted_keys,redis.googleapis.com/stats/expired_keys,redis.googleapis.com/stats/memory/maxmemory,redis.googleapis.com/stats/memory/usage,redis.googleapis.com/stats/memory/usage_ratio,redis.googleapis.com/stats/network_traffic,redis.googleapis.com/stats/reject_connections_count
Logs
time="2019-10-17T19:27:52Z" level=info msg="Starting stackdriver_exporter (version=0.6.0, branch=HEAD, revision=59b7f38fa3d7e7ff8fd7134dd373c91051d5c16c)" source="stackdriver_exporter.go:121"
time="2019-10-17T19:27:52Z" level=info msg="Build context (go=go1.11.2, user=root@28c8704f4734, date=20181202-06:28:44)" source="stackdriver_exporter.go:122"
time="2019-10-17T19:27:52Z" level=info msg="Listening on :9255" source="stackdriver_exporter.go:148"
Exit Code
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Thu, 17 Oct 2019 21:27:52 +0200
      Finished:     Thu, 17 Oct 2019 21:28:20 +0200
    Ready:          False
    Restart Count:  10

elsbrock avatar Oct 17 '19 19:10 elsbrock