dnsbl_exporter
dnsbl_exporter copied to clipboard
Luzilla
upload example dashboard (grafana)
No dashboard(s) yet? How do you guys use this? Only set alerts in alertmanager? examples? thanks!
Same. But happy to take contributions. Will try to roll a new release this month as well.
great to hear. Could one of you share the alert definition? Or even add it to the README? ;-) thanks
great to hear. Could one of you share the alert definition? Or even add it to the README? ;-) thanks
It's just the usual:
- alert: MxListed
expr: readme query example
for: 1m or your scrape interval
labels:
prio: page
annotations:
summary: ...
description: ...
The readme has a query example already. Does that help? If you'd improve on it, I'd be happy to merge too. :)
great to hear. Could one of you share the alert definition? Or even add it to the README? ;-) thanks
It's just the usual:
- alert: MxListed expr: readme query example for: 1m or your scrape interval labels: prio: page annotations: summary: ... description: ...The readme has a query example already. Does that help? If you'd improve on it, I'd be happy to merge too. :)
;-) I see. I'll try maybe.
Writing my alerts now ...
question: in the query, what is the IP?
cat dnsbl_exporter.yml
groups:
- name: dnsbl_exporter
rules:
- alert: MxListed
expr: luzilla_rbls_ips_blacklisted{hostname="co.oops.co.at",ip="45.84.138.128",rbl="ix.dnsbl.manitu.net"}
for: 1m
Do I have to list both FQDN and IP? Wouldn't it be able to query DNS for that? Or do I misunderstand? thanks.
@stefangweichinger if you look at the data in Prometheus, it gets annotated with an IP, as a DNS record could resolve to multiple IPs.
So technically you don't need to query with it unless you have multiple IPs and want different alerting rules (or easier to understand alerts).
But generally, my code probes for if-hostname and attempts to resolve all IPs that are behind it, and then goes through the list of IPs and checks the RBLs, therefor, an IP is added as a dimension/label added.
I could also see the labels being useful in a dashboard setting if you want a panel by RBL that shows hits/no hits. Etc..
Btw, you probably want an expr like:
expr: luzilla_rbls_ips_blacklisted{hostname="co.oops.co.at"} > 0
But again, enter the query in Grafana/prometheus UI and see the results. A 1 designates it's listed in an RBL.
@till great, thanks. I understand so far and will be able to use that as mentioned.
So far I am stuck at a lower level: I have prometheus in docker and your exporter as systemd-service on the host. I get metrics on host-level, but somehow prometheus doesn't yet scrape them. This is off-topic regarding this thread's topic, so I shut up here ;-) (yes, I use something like "host.docker.internal" in docker-compose etc etc ... works for other exporters ...)
If you want to create a new ticket, maybe we can figure it out. I mean, you could also run the exporter in a container, but not sure what difference it makes.
@till Thanks, I might do so later. Right now busy ... / and I assume it's more of a docker-issue in my place. btw your Dockerfile fails and there are no images in the mentioned registry ;-) / I assume you know this. No complaint, only feedback.
If you want to create a new ticket, maybe we can figure it out. I mean, you could also run the exporter in a container, but not sure what difference it makes.
opened #177 . thanks
In a second installation I run prometheus on a host (and not in docker) and there I get metrics into prometheus. But so far no metric called luzilla_rbls_ips_blacklisted as mentioned in your example above.
I get:
# curl http://localhost:9211/metrics
# HELP luzilla_rbls_duration The scrape's duration (in seconds)
# TYPE luzilla_rbls_duration gauge
luzilla_rbls_duration 0.000881381
# HELP luzilla_rbls_listed The number of listings in RBLs (this is bad)
# TYPE luzilla_rbls_listed gauge
luzilla_rbls_listed{rbl="ix.dnsbl.manitu.net"} 0
luzilla_rbls_listed{rbl="pbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="sbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="xbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="zen.spamhaus.org"} 0
# HELP luzilla_rbls_targets The number of targets that are being probed (configured via targets.ini or ?target=)
# TYPE luzilla_rbls_targets gauge
luzilla_rbls_targets 5
# HELP luzilla_rbls_used The number of RBLs to check IPs against (configured via rbls.ini)
# TYPE luzilla_rbls_used gauge
luzilla_rbls_used 5
# HELP promhttp_metric_handler_errors_total Total number of internal errors encountered by the promhttp metric handler.
# TYPE promhttp_metric_handler_errors_total counter
promhttp_metric_handler_errors_total{cause="encoding"} 0
promhttp_metric_handler_errors_total{cause="gathering"} 0
Might be related to:
# journalctl -f -u dnsbl-exporter.service
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.005+01:00 level=ERROR msg="read udp 127.0.0.1:39764->:0: read: connection refused" area=metrics target=co.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.005+01:00 level=ERROR msg="read udp 127.0.0.1:38256->:0: read: connection refused" area=metrics target=co.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.005+01:00 level=ERROR msg="read udp 127.0.0.1:33010->:0: read: connection refused" area=metrics target=co.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.005+01:00 level=ERROR msg="read udp 127.0.0.1:52243->:0: read: connection refused" area=metrics target=co.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.005+01:00 level=ERROR msg="read udp 127.0.0.1:55305->:0: read: connection refused" area=metrics target=co.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.006+01:00 level=ERROR msg="read udp 127.0.0.1:33459->:0: read: connection refused" area=metrics target=oc.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.006+01:00 level=ERROR msg="read udp 127.0.0.1:41222->:0: read: connection refused" area=metrics target=oc.oops.co.at
Nov 16 19:55:05 mail2 dnsbl-exporter[1144049]: time=2023-11-16T19:55:05.006+01:00 level=ERROR msg="read udp 127.0.0.1:60388->:0: read: connection refused" area=metrics target=oc.oops.co.at
whatever that means ...
The server with the exporter is in a DMZ, Prometheus is in another subnet, but allowed to scrape the port 9211.
I see 4 metrics in Grafana:
- luzilla_rbls_duration
- luzilla_rbls_listed
- luzilla_rbls_targets
- luzilla_rbls_used
I should split that into a separate issue, right? off-topic in terms of the issue subject. I understand that there is some connection issue, but I don't understand what fails. The dns-resolver IP is OK and works.
level=ERROR msg="read udp 127.0.0.1:41114->:0: read: connection refused" area=metrics target=oc.oops.co.at
What is tried here? What is ":0:" ? I assume 41114 is a random source port?
I think the readme is just wrong, it's listed, not blacklisted. 😅
The other problem, I am not sure.
I think the readme is just wrong, it's listed, not blacklisted. 😅
What exactly? I don't understand.
The other problem, I am not sure.
oh ...
Do you understand that ":0:" ? Is that a port? I assume the binary doesn't get DNS replies or so.
similar log-lines mentioned in https://github.com/Luzilla/dnsbl_exporter/issues/64#issuecomment-817149757
The machine is in a DMZ, and is configured to use the DNS resolver on the gateway, which is a pfSense running unbound. I checked the firewall rules, UDP is allowed on port 53. And the resolving works on the shell.
For debugging I also allowed TCP now and restarted dnsbl_exporter: same errors.
The mentioned issue above says that the exporter worked correctly anyways. Not in my case.
All the metrics I currently get:
root@mail2:~# curl http://127.0.0.1:9211/metrics
# HELP luzilla_rbls_duration The scrape's duration (in seconds)
# TYPE luzilla_rbls_duration gauge
luzilla_rbls_duration 0.001042781
# HELP luzilla_rbls_listed The number of listings in RBLs (this is bad)
# TYPE luzilla_rbls_listed gauge
luzilla_rbls_listed{rbl="ix.dnsbl.manitu.net"} 0
luzilla_rbls_listed{rbl="pbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="sbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="xbl.spamhaus.org"} 0
luzilla_rbls_listed{rbl="zen.spamhaus.org"} 0
# HELP luzilla_rbls_targets The number of targets that are being probed (configured via targets.ini or ?target=)
# TYPE luzilla_rbls_targets gauge
luzilla_rbls_targets 5
# HELP luzilla_rbls_used The number of RBLs to check IPs against (configured via rbls.ini)
# TYPE luzilla_rbls_used gauge
luzilla_rbls_used 5
# HELP promhttp_metric_handler_errors_total Total number of internal errors encountered by the promhttp metric handler.
# TYPE promhttp_metric_handler_errors_total counter
promhttp_metric_handler_errors_total{cause="encoding"} 0
promhttp_metric_handler_errors_total{cause="gathering"} 0
The other problem, I am not sure.
Solved on the host with prometheus in docker: there was an iptables-rule missing for port 9211 ... the server uses ferm ...
I moved the exporter from the mail-server in the DMZ to the prometheus-server in LAN, and edited the dns-resolve. Same behavior: only the mentioned metrics are returned. Do I have to wait for a certain amount of time?
@stefangweichinger Can you try the latest release. There were some fixes in main that I just got around to releasing.
@till sorry, this slipped through. Will test asap, very likely on monday. thanks
Ah, so it's tuesday ;-) Upgraded/replaced the script. I still need the firewall-rule in place for the prometheus-container to access the exporter on the docker host. But I get metrics into Prometheus, yes. Will look through the latest changes now, thanks.
@stefangweichinger @zentavr — I re-did one of my internal dashboards and uploaded it in #219 — not entirely sure if it works when you import it. But if either of you wants to help test and give me feedback, much appreciated. I'll also see if I can add it to Grafana.com.
Oh, a dozen will probably not look too nice. :D
Which ones are you using? I'll add them later to my config. I've gotten by with these two mostly over the years.
sum(luzilla_rbls_listed{}) by(rbl)
@till:
# 0SPAM
- bl.0spam.org
- rbl.0spam.org
# NIXSPAM
- ix.dnsbl.manitu.net
# Spamhaus Zen
- zen.spamhaus.org
# Abusix Exploit Blacklist
- <your_id_here>.exploit.mail.abusix.zone
# Abusix Domain Blacklist
- <your_id_here>.dblack.mail.abusix.zone
# Abusix Spam Blacklist
- <your_id_here>.black.mail.abusix.zone
# Anonmails DNSBL
- spam.dnsbl.anonmails.de
# UCEPROTECT-Level 1
- dnsbl-1.uceprotect.net
# UCEPROTECT-Level 2
- dnsbl-2.uceprotect.net
# UCEPROTECT-Level 2
- dnsbl-3.uceprotect.net
# Backscatterer
- ips.backscatterer.org
# Barracuda Reputation Block List
- b.barracudacentral.org
# Blocklist.de
- bl.blocklist.de
# CALIVENT
- dnsbl.calivent.com.pe
# CYMRU BOGONS
- bogons.cymru.com
# DNS Servicios
- rbl.dns-servicios.com
# DRMX
- bl.drmx.org
# DRONEBL
- dnsbl.dronebl.org
# FABEL SOURCES
- spamsources.fabel.dk
# HIL HABEAS
- hil.habeas.com
# HIL2 HABEAS
- hil2.habeas.com
# Hostkarma
- hostkarma.junkemailfilter.com
# IBM DNS Blacklist
- dnsbl.cobion.com
# ICM FORBIDDEN
- forbidden.icm.edu.pl
# IMP WORM
- dnsrbl.swinog.ch
# IMP SPAM
- spamrbl.swinog.ch
- uribl.swinog.ch
# Spamhaus ZEN
- zen.spamhaus.org
# Spamhaus DBL - should not be used with IPs
#- dbl.spamhaus.org
- xbl.spamhaus.org
# SPFBL DNSBL
- dnsbl.spfbl.net
# Sender Score Reputation Network
- bl.score.senderscore.com
# SORBS BLOCK
- block.dnsbl.sorbs.net
# SORBS DUHL
- dul.dnsbl.sorbs.net
# SORBS HTTP
- http.dnsbl.sorbs.net
# SORBS MISC
- misc.dnsbl.sorbs.net
# SORBS NEW
- new.spam.dnsbl.sorbs.net
# SORBS SMTP
- smtp.dnsbl.sorbs.net
# SORBS SOCKS
- socks.dnsbl.sorbs.net
# SORBS SPAM
- spam.dnsbl.sorbs.net
# SORBS WEB
- web.dnsbl.sorbs.net
# SORBS ZOMBIE
- zombie.dnsbl.sorbs.net
# RATS Dyna
- dyna.spamrats.com
# RATS NoPtr
- noptr.spamrats.com
# RATS Spam
- spam.spamrats.com
# SEM BACKSCATTER
- backscatter.spameatingmonkey.net
# SEM BLACK
- bl.spameatingmonkey.net
# MSRBL Phishing
- phishing.rbl.msrbl.net
# MSRBL Spam
- spam.rbl.msrbl.net
# NETHERRELAYS
- relays.nether.net
# NETHERUNSURE
- unsure.nether.net
# NIXSPAM
- ix.dnsbl.manitu.net
# Nordspam BL
- bl.nordspam.com
# NoSolicitado
- bl.nosolicitado.org
# ORVEDB
- orvedb.aupads.org
# PSBL
- psbl.surriel.com
# RBL JP
- virus.rbl.jp
# RSBL
- rsbl.aupads.org
# s5h.net
- all.s5h.net
# SCHULTE
- rbl.schulte.org
# SERVICESNET
- korea.services.net
# SPAMCOP
- bl.spamcop.net
# Suomispam Reputation
- bl.suomispam.net
# SWINOG
- dnsrbl.swinog.ch
# TRIUMF
- rbl2.triumf.ca
# TRUNCATE
- truncate.gbudb.net
# Woodys SMTP Blacklist
- blacklist.woody.ch
# WPBL
- db.wpbl.info
# ZapBL
- dnsbl.zapbl.net
# INTERSERVER
- rbl.interserver.net
# JIPPG
- dialup.blacklist.jippg.org
# KEMPTBL
- dnsbl.kempt.net
# KISA
- spamlist.or.kr
# Konstant
- bl.konstant.no
# LASHBACK
- ubl.lashback.com
# LNSGBLOCK
- spamguard.leadmon.net
# MADAVI
- dnsbl.madavi.de
# MAILSPIKE BL
- bl.mailspike.net
# MAILSPIKE Z
- z.mailspike.net
...kinda mx.toolbox.com - The list is taken from there + Google + research