azure-metrics-exporter icon indicating copy to clipboard operation
azure-metrics-exporter copied to clipboard

Unable to extract metrics from managed cluster (AKS)

Open Giri0422 opened this issue 2 years ago • 5 comments

i tried deploy this on my aks cluster and i was able to extract metrics from resourcetype like keyvault.storageaccount,redis etc. when i did the same for Microsoft.ContainerService/managedClusters resourcetype i am not getting any results i have assigned monitoring reader role as well. can you please let me know if this exporter supports managed cluster resource type or am i wrong. can you please help on this.

Job_config :

- job_name: azure-metrics-ContainerService
        scrape_interval: 1m
        metrics_path: /probe/metrics/list
        params:
          name: ["azure-metrics-ContainerService"]
          subscription:
          - "{{ .Values.subscriptionId }}"
          filter: ["(resourceGroup eq '{{ .Values.resourceGroup }}' or resourceGroup eq '{{ .Values.resourceGroup }}-aks') and resourceType eq 'Microsoft.ContainerService/managedClusters'"]
          metrics:
          - apiserver_current_inflight_requests
          - cluster_autoscaler_cluster_safe_to_autoscale
          - cluster_autoscaler_scale_down_in_cooldown
          - cluster_autoscaler_unneeded_nodes_count
          - cluster_autoscaler_unschedulable_pods_count
          - kube_node_status_allocatable_cpu_cores
          - kube_node_status_allocatable_memory_bytes
          - kube_node_status_condition
          - kube_pod_status_phase
          - kube_pod_status_ready
          - node_cpu_usage_millicores
          - node_cpu_usage_percentage
          - node_disk_usage_bytes
          - node_disk_usage_percentage
          - node_memory_rss_bytes
          - node_memory_rss_percentage
          - node_memory_working_set_bytes
          - node_memory_working_set_percentage
          - node_network_in_bytes
          - node_network_out_bytes
          interval: ["PT1M"]
          timespan: ["PT1M"]
          aggregation:
          - average
          - total
        static_configs:
        - targets: ["azure-metrics-exporter:8080"]

Giri0422 avatar Mar 24 '23 10:03 Giri0422

it actually supports all metrics which you can also get via the Azure Portal (it's just a client for the Monitor API, nothing more).

What error do you get or do you just get nothing? Which version are you using? Have you tried the query page for testing (http://azure-metrics-exporter:8080/query)?

mblaschke avatar Mar 28 '23 23:03 mblaschke

@mblaschke : i am using version 1.0.2. i dont see any error. yes i tried query page too still no luck. what i obserrved was i was able to see metrics by specifying the target of a managed cluster but doesnt work on service discovery

using single instance something like this worked but no ServiceDiscovery

 - job_name: azure-metrics-redis
  scrape_interval: 1m
  metrics_path: /probe/metrics/resource
  params:
    name: ["my_own_metric_name"]
    subscription:
    - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    target:
    - /subscriptions/.../resourceGroups/.../providers/Microsoft.Cache/Redis/...
    - /subscriptions/.../resourceGroups/.../providers/Microsoft.Cache/Redis/...
    - /subscriptions/.../resourceGroups/.../providers/Microsoft.Cache/Redis/...
    - /subscriptions/.../resourceGroups/.../providers/Microsoft.Cache/Redis/...
    metric:
    - connectedclients
    - totalcommandsprocessed
    - cachehits
    - cachemisses
    - getcommands
    - setcommands
    - operationsPerSecond
    - evictedkeys
    - totalkeys
    - expiredkeys
    - usedmemory
    - usedmemorypercentage
    - usedmemoryRss
    - serverLoad
    - cacheWrite
    - cacheRead
    - percentProcessorTime
    - cacheLatency
    - errors
    interval: ["PT1M"]
    timespan: ["PT1M"]
    aggregation:
    - average
    - total
  static_configs:
  - targets: ["azure-metrics:8080"]

Giri0422 avatar May 17 '23 06:05 Giri0422

@mblaschke could you please help on this!

Giri0422 avatar May 29 '23 09:05 Giri0422

@Giri0422 what do you mean with version 1.0.2? there is no version 1.0.2 for azure-metrics-exporter 🤔

you changed the metric url when using service endpoint? i recommend using Azure ResourceGraph for ServiceDiscovery as it's the fastest way.

with which config have you tried ServiceConfig?

mblaschke avatar Jul 01 '23 23:07 mblaschke

it actually supports all metrics which you can also get via the Azure Portal (it's just a client for the Monitor API, nothing more).

What error do you get or do you just get nothing? Which version are you using? Have you tried the query page for testing (http://azure-metrics-exporter:8080/query)?

What exactly is meant by HTTP 0 error?

jiriwetter avatar Apr 30 '24 08:04 jiriwetter