An error prometheus scrape config will not cause agent error

[liuyangc3]

promtheus config example

# error.yml
global:
  scrape_interval: 1m
  scrape_timeout: 10s
scrape_configs:
  - job_name: docker
    docker_sd_config:
      - host: unix:///var/run/docker.sock
        filters:
          - name: label
            value:
              - aaa=bbb

this config is not a correct config docker_sd_config should be docker_sd_configs and filters value should be values even the promtool can report the error.

$ promtool check config error.yml
Checking error.yml
  FAILED: parsing YAML file error.yml: yaml: unmarshal errors:
  line 6: field docker_sd_config not found in type config.ScrapeConfig

Checking error.yml
  FAILED: parsing YAML file error.yml: yaml: unmarshal errors:
  line 10: field value not found in type moby.Filter

but cwagent will start successfully without errors, which makes the users hard to debug some issues

2021/07/21 10:06:26 I! I! Detected the instance is EC2
2021/07/21 10:06:26 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json does not exist or cannot read. Skipping it.
2021/07/21 10:06:26 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_cwagent-config.json ...
Valid Json input schema.
I! Detecting run_as_user...
No csm configuration found.
No metric configuration found.
Configuration validation first phase succeeded

2021/07/21 10:06:26 I! Config has been translated into TOML /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml
2021/07/21 10:06:26 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
2021/07/21 10:06:26 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_cwagent-config.json ...
2021/07/21 10:06:26 I! Detected runAsUser: root
2021/07/21 10:06:26 I! Changing ownership of [/opt/aws/amazon-cloudwatch-agent/logs /opt/aws/amazon-cloudwatch-agent/etc /opt/aws/amazon-cloudwatch-agent/var] to 0:0
2021-07-21T10:06:26Z I! Starting AmazonCloudWatchAgent 1.247348.0
2021-07-21T10:06:26Z I! Loaded inputs: prometheus_scraper
2021-07-21T10:06:26Z I! Loaded aggregators:
2021-07-21T10:06:26Z I! Loaded processors: emfProcessor
2021-07-21T10:06:26Z I! Loaded outputs: cloudwatchlogs
2021-07-21T10:06:26Z I! Tags enabled: host=ec2
2021-07-21T10:06:26Z I! [agent] Config: Interval:1m0s, Quiet:false, Hostname:"ec2", Flush Interval:1s
2021-07-21T10:06:26Z D! [agent] Initializing plugins
2021-07-21T10:06:26Z D! [agent] Connecting outputs
2021-07-21T10:06:26Z D! [agent] Attempting connection to [outputs.cloudwatchlogs]
2021-07-21T10:06:26Z D! [agent] Successfully connected to outputs.cloudwatchlogs
2021-07-21T10:06:26Z D! [agent] Starting service inputs
2021-07-21T10:06:26Z I! [logagent] starting
2021-07-21T10:06:26Z I! [logagent] found plugin cloudwatchlogs is a log backend

[jhnlsn]

Thanks for submitting this request. I will check in with the team to see if this is something that we want to support in-agent

[Dem0n3D]

Just faced this issue. I don't know golang, but I think cause in this lines. Docker repo has been moved to https://github.com/moby/moby.