prometheus and splunk intergration-

[Davinder2609]

TYPE confluent_kafka_server_request_bytes gauge

[Davinder2609]

used below configration scrape_configs:

• job_name: Confluent Cloud scrape_interval: 1m scrape_timeout: 1m honor_timestamps: true static_configs:
  • targets:
    • api.telemetry.confluent.cloud scheme: https basic_auth: username: <Cloud API Key> password: <Cloud API Secret> metrics_path: /v2/metrics/cloud/export params: "resource.kafka.id":
    • lkc-1
    • lkc-2

[Davinder2609]

In Graph I can see all mertics . but localhost/9090 I am not able to see confluent_kafka_server_received_bytes confluent_kafka_server_sent_bytes confluent_kafka_server_received_records confluent_kafka_server_sent_records confluent_kafka_server_retained_bytes confluent_kafka_server_request_count confluent_kafka_server_active_connection_count confluent_kafka_server_partition_count

[Davinder2609]

@Dabz Please advise on this issue

[Dabz]

Hi @Davinder2609 , as mentioned in the README, this project is no longer maintained and I encourage switching to the Confluent Cloud export endpoint that expose OpenMetrics data.

I would be glad to help if required, but I struggle to understand the problem, if you are able to see those metrics in Prometheus, it indicates a successful fetch & scrape.

[Davinder2609]

Thanks @Dabz ,with ccloud_exporter we are able to get all cloud metrics in splunk.but with promethueus we are not getting all confluent cloud metrics in splunk,only scrape_duration_seconds,scrape_samples_post_metric_relabeling,scrape_samples_scraped,scrape_series_added listed in splunk. below mertics not listed- confluent_kafka_server_received_bytes confluent_kafka_server_sent_bytes confluent_kafka_server_received_records confluent_kafka_server_sent_records confluent_kafka_server_retained_bytes confluent_kafka_server_request_count confluent_kafka_server_active_connection_count confluent_kafka_server_partition_count

[Dabz]

Check the ccloudexporter logs, but very likely something is failing. The metrics you list are the one that are always returned, regardless of the success of the queries to the Metrics API.

[Davinder2609]

We are not using ccloudexporter,Only using Receiver- promethues Exporter- splunk_hec/metrics

[Davinder2609]

using these 2 files

otel-collector-config.yml

receivers: prometheus_simple: collection_interval: 60s endpoint: prometheus:9090

   metrics_path: /federate

   params:
    match[]:
      - '{job="ccloud_metrics"}'
      - '{__name__=~"job:.*"}'

otlp:
    protocols:
        grpc:
          endpoint: 0.0.0.0:4317
        http:
          endpoint: 0.0.0.0:4318

exporters: splunk_hec/metrics: # Splunk HTTP Event Collector token. token: "************************"

    # URL to a Splunk instance to send data to.
    endpoint: "https://*****splunkcloud.com:8088/services/collector"
    source: "http:kafkademo"
    sourcetype: "manual"
    index: "main"
    max_connections: 20
    disable_compression: false
    timeout: 10s      
    insecure_skip_verify: true

processors: batch:

extensions: health_check: endpoint: 0.0.0.0:13133 pprof: endpoint: :1888 zpages: endpoint: :55679

service: extensions: [pprof, zpages, health_check] pipelines: metrics: receivers: [prometheus_simple] processors: [batch] exporters: [splunk_hec/metrics]

Prometheus.yml

scrape_configs:

• job_name: Confluent Cloud scrape_interval: 60s honor_labels: true honor_timestamps: true

  static_configs:

  • targets:
    • api.telemetry.confluent.cloud

  scheme: https basic_auth: username: ********* password: *************

  metrics_path: /v2/metrics/cloud/export

  params: resource.kafka.id: - ********** resource.schema_registry.id: - **********

  tls_config: insecure_skip_verify: true

[Davinder2609]

@Dabz Please advise!