swarmprom icon indicating copy to clipboard operation
swarmprom copied to clipboard

Prometheus not connecting to alertmanager

Open Power2All opened this issue 6 years ago • 4 comments

I got this problem that prometheus isn't connecting to alertmanager. Network looks to be fine, but it's not able to connect to the IP address given in the logs.

level=info ts=2019-03-26T10:57:47.979177541Z caller=main.go:244 msg="Starting Prometheus" version="(version=2.5.0, branch=HEAD, revision=67dc912ac8b24f94a1fc478f352d25179c94ab9b)",
level=info ts=2019-03-26T10:57:47.982924182Z caller=main.go:245 build_context="(go=go1.11.1, user=root@578ab108d0b9, date=20181106-11:40:44)",
level=info ts=2019-03-26T10:57:47.982956117Z caller=main.go:246 host_details="(Linux 4.15.18-11-pve #1 SMP PVE 4.15.18-34 (Mon, 25 Feb 2019 14:51:06 +0100) x86_64 227ce4d1d192 (none))",
level=info ts=2019-03-26T10:57:47.983021401Z caller=main.go:247 fd_limits="(soft=1048576, hard=1048576)",
level=info ts=2019-03-26T10:57:47.983047421Z caller=main.go:248 vm_limits="(soft=unlimited, hard=unlimited)",
level=info ts=2019-03-26T10:57:47.983826146Z caller=main.go:562 msg="Starting TSDB ...",
level=info ts=2019-03-26T10:57:47.988462388Z caller=web.go:399 component=web msg="Start listening for connections" address=0.0.0.0:9090,
level=info ts=2019-03-26T10:57:48.003159108Z caller=main.go:572 msg="TSDB started",
level=info ts=2019-03-26T10:57:48.003214832Z caller=main.go:632 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml,
level=info ts=2019-03-26T10:57:48.01808873Z caller=main.go:658 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml,
level=info ts=2019-03-26T10:57:48.018152796Z caller=main.go:531 msg="Server is ready to receive web requests.",
level=error ts=2019-03-26T10:57:56.045698965Z caller=notifier.go:473 component=notifier alertmanager=http://alertmanager:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://alertmanager:9093/api/v1/alerts: dial tcp 10.0.6.7:9093: connect: no route to host",
level=error ts=2019-03-26T10:59:06.157660555Z caller=notifier.go:473 component=notifier alertmanager=http://alertmanager:9093/api/v1/alerts count=1 msg="Error sending alert" err="Post http://alertmanager:9093/api/v1/alerts: dial tcp 10.0.6.7:9093: connect: no route to host",

Alertmanager runs fine:

image image

Power2All avatar Mar 26 '19 11:03 Power2All

Nobody seem to be able to figure out this problem ?

Power2All avatar Apr 30 '19 12:04 Power2All

same problem, any help?

xkfen avatar Apr 08 '20 04:04 xkfen

having the same issue :(

mashun4ek avatar Feb 04 '21 00:02 mashun4ek

seems like you containers are not in the same network, just attach your alermanager container to the same network as your prometheus and it should work

aduzsardi avatar Feb 04 '21 07:02 aduzsardi