is there a CRD that can set ec2_sd_config?

[alexandrst88]

[f41gh7]

No, there is no such CRD. It's possible to define it with inline raw configuration at the VMAgent section:

kind: VMAgent
spec:
  inlineScrapeConfig: |
   - job_name: ec2-monitoring
     ec2_sd_configs:
     - region: eu-west-1
       port: 9100

Hm, maybe it's possible to create some generic CRD for all other service discovery types. Need to think about it.

[lacer-utkarsh]

I get this error when I try to set the inlineScrapeConfig:

2022-05-16T18:13:12.748Z	error	VictoriaMetrics/lib/promscrape/config.go:829	skipping ec2_sd_config targets for job_name "my_test_ec2" because of error: error when fetching instances data from EC2: cannot obtain instances: cannot obtain fresh credentials for EC2 API: cannot get instanceRoleName: cannot obtain IMDSv2 session token from "http://169.254.169.254/latest/api/token"; probably, `region` is missing in `ec2_sd_config`; error: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

This is my config:

inlineScrapeConfig: |
        - job_name: "my_test_ec2"
          ec2_sd_configs:
            - region: us-west-2
              port: 9644

[Vladyslav-Miletskyi]

@lacer-utkarsh

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-PrometheusEC2.html

The IAM role for the EC2 instance where the CloudWatch agent runs must include the ec2:DescribeInstance permission. For example, you could attach the managed policy AmazonEC2ReadOnlyAccess to the instance running the CloudWatch agent.

make sure, that all k8s nodes have IAM role with ec2:DescribeInstance policy allowed

[Vladyslav-Miletskyi]

Scrape EC2s, which are tagged with prometheus.io/scrape=true

  inlineScrapeConfig: |
    - job_name: ec2
      honor_labels: true
      ec2_sd_configs:
        - region: us-west-1
          filters:
          - name: tag:prometheus.io/scrape
            values:
              - "true"
      relabel_configs:
        - source_labels: [__meta_ec2_tag_prometheus_io_path]
          target_label: __metrics_path__
        - source_labels: [__address__, __meta_ec2_tag_prometheus_io_port]
          action: replace
          regex: ([^:]+)(?::\d+)?;(\d+)
          replacement: $1:$2
          target_label: __address__
        - source_labels: [__meta_ec2_tag_prometheus_io_job]
          target_label: job
        - regex: __meta_ec2_(instance_id|availability_zone|instance_type)
          action: labelmap
        - regex: __meta_ec2_tag_prometheus_io(.+)
          action: labeldrop
        - action: labelmap
          regex: __meta_ec2_tag_(.+)

[mike2194]

No, there is no such CRD. It's possible to define it with inline raw configuration at the VMAgent section:

kind: VMAgent
spec:
  inlineScrapeConfig: |
   - job_name: ec2-monitoring
     ec2_sd_configs:
     - region: eu-west-1
       port: 9100

Hm, maybe it's possible to create some generic CRD for all other service discovery types. Need to think about it.

any thoughts on this? I would very much like to move away from inlineScrapeConfig so i can generate scrape configs and validate them via helm/kubectl