restart often collapse

[jackyliusohu]

restart often

• Prometheus Operator version: app.kubernetes.io/version: v0.40.0 spec: containers: - args: - --kubelet-service=kube-system/kubelet - --logtostderr=true - --config-reloader-image=harbor.weizhipin.com/prom/configmap-reload:v0.3.0 - --prometheus-config-reloader=harbor.weizhipin.com/prom/prometheus-config-reloader:v0.40.0 image: harbor.weizhipin.com/prom/prometheus-operator:v0.40.0

• Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

• Prometheus Operator Logs: level=warn ts=2022-11-09T00:03:21.172272303Z caller=promcfg.go:494 component=prometheusoperator msg="PodMonitor 'targetPort' is deprecated, use 'port' instead." podMonitor=ebpf-exporter level=warn ts=2022-11-09T00:03:21.172321614Z caller=promcfg.go:494 component=prometheusoperator msg="PodMonitor 'targetPort' is deprecated, use 'port' instead." podMonitor=kindling level=info ts=2022-11-09T00:03:21.189796311Z caller=operator.go:1122 component=prometheusoperator msg="sync prometheus" key=monitoring/k8s-business-d-b level=info ts=2022-11-09T00:03:21.420617936Z caller=operator.go:1122 component=prometheusoperator msg="sync prometheus" key=monitoring/k8s-business-e-b level=info ts=2022-11-09T00:03:21.68040675Z caller=operator.go:1122 component=prometheusoperator msg="sync prometheus" key=monitoring/k8s-business-g^C

Insert Prometheus Operator logs relevant to the issue here

• Prometheus Logs: Args: --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/console_libraries --config.file=/etc/prometheus/config_out/prometheus.env.yaml --storage.tsdb.path=/prometheus --storage.tsdb.retention.time=2h --web.enable-lifecycle --storage.tsdb.no-lockfile --web.route-prefix=/ State: Running Started: Sat, 10 Dec 2022 17:22:30 +0800 Last State: Terminated Reason: Error Message: level=error ts=2022-12-10T09:22:28.924Z caller=main.go:350 msg="Error loading config (--config.file=/etc/prometheus/config_out/prometheus.env.yaml)" err="open /etc/prometheus/config_out/prometheus.env.yaml: no such file or directory"

[hakuna-matatah]

Facing the same issue:

 Message:   level=error ts=2023-12-12T12:32:46.718Z caller=main.go:333 msg="Error loading config (--config.file=/etc/prometheus/config_out/prometheus.env.yaml)" err="open /etc/prometheus/config_out/prometheus.env.yaml: no such file or directory"