Hope to add ScrapeConfig CRD

[linuxduduniao]

Related functions have been provided by Prometheus Operator ScrapeConfig

[f41gh7]

@Amper @Haleygo WDYT?

[Haleygo]

Yeah, I think we can implement it as a generic scrape configuration for all service discoveries.

[linuxduduniao]

Must support sharding for ScrapeConfig

[Haleygo]

Must support sharding for ScrapeConfig

Can you elaborate here? You mean to shard targets from one scrape job to several vmagents?

[linuxduduniao]

Must support sharding for ScrapeConfig

Can you elaborate here? You mean to shard targets from one scrape job to several vmagents?

For example, when reading the instances address from consul (node_exporter, icmp, applications deployed on virtual machines), when there are many instances, I need to use sharding to solve a single vmagent performance issue.

[linuxduduniao]

I want deploy vmagents on Kubernetes, these vmagents scrape all targets within the cluster( serverMonitor, podMonitor ...) and outside the cluster(scrapeConfig). Management CRD configuration by gitops (argocd + gitlab)

[Haleygo]

For example, when reading the instances address from consul (node_exporter, icmp, applications deployed on virtual machines), when there are many instances, I need to use sharding to solve a single vmagent performance issue.

This is already implemented on vmagent by shardCount, see https://new.docs.victoriametrics.com/operator/resources/vmagent/?highlight=shardcount#sharding. ScrapeConfig CRD will be a config resource like other scrape CRDs, help generating scrape_config file.

[linuxduduniao]

This is already implemented on vmagent by shardCount, see https://new.docs.victoriametrics.com/operator/resources/vmagent/?highlight=shardcount#sharding. ScrapeConfig CRD will be a config resource like other scrape CRDs, help generating scrape_config file.

Thanks. Because prometheus operator not support sharding for ScrapeConfig , so I wanted vm operator support sharding for ScrapeConfig

[f41gh7]

Must be fixed at v0.43.0 release

Operator now supports ScrapeConfig and implements own version with VMScrapeConfig