${subsystem}_responses_sent metic resets during "planned periodic restart of DNS server"

[darrencloutier]

Hello,

Thanks again for the amazing DNS project. Just wanted to comment on a low priority issue (maybe it's by design?) that seems to have started with commit 6a383ba of v2.0.0-beta.

During the daily "planned periodic restart of DNS server" the ${subsystem}_responses_sent metric now resets to 0. Previously this was persisted during these restarts and only a restart of the full container would reset this number.

Logs are below from my setup. Every 24 hours during that planned restart this metric is resetting.

Thanks again!

========================================
========================================
================= dns ==================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version v2.0.0-beta built on 2022-06-27T14:07:18Z (commit 6a383ba)

🔧 Need help? https://github.com/qdm12/dns/discussions/new
🐛 Bug? https://github.com/qdm12/dns/issues/new
✨ New feature? https://github.com/qdm12/dns/issues/new
☕ Discussion? https://github.com/qdm12/dns/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2022-09-25T17:32:13Z INFO Settings:
├── DNS upstream connection: dot
├── DNS server listening address: :53
├── DNS over TLS:
|   ├── DNS over TLS providers: cloudflare and google
|   ├── Plaintext fallback DNS providers: cloudflare
|   ├── Request timeout: 3s
|   └── Connecting over: IPv4
├── Cache:
|   ├── Type: lru
|   └── LRU cache:
|       └── Max entries: 10000
├── Filtering:
|   ├── Blocked categories: 
|   └── Rebinding protection: enabled
├── Logging:
|   └── Level: INFO
├── Middleware logging: disabled
├── Metrics:
|   ├── Type: prometheus
|   └── Prometheus:
|       ├── Listening address: :9090
|       └── Subsystem: dns3
├── Check DNS: enabled
└── Periodic update: every 24h0m0s
2022-09-25T17:32:13Z INFO using DNS address 127.0.0.1 internally
2022-09-25T17:32:13Z INFO [prometheus server] listening on :9090
2022-09-25T17:32:13Z INFO [healthcheck server] listening on 127.0.0.1:9999
2022-09-25T17:32:13Z INFO starting DNS server
2022-09-25T17:32:13Z INFO DNS server listening on :53
2022-09-25T17:32:13Z INFO restarting DNS server the first time to get updated files
2022-09-25T17:32:13Z INFO downloading and building DNS block lists
2022-09-25T17:32:13Z INFO 0 hostnames blocked overall
2022-09-25T17:32:13Z INFO 0 IP addresses blocked overall
2022-09-25T17:32:13Z INFO 13 IP networks blocked overall
2022-09-25T17:32:13Z INFO starting DNS server
2022-09-25T17:32:13Z INFO DNS server listening on :53
2022-09-26T13:13:21Z WARN dial tcp 1.1.1.1:853: i/o timeout
2022-09-26T17:01:16Z WARN dial tcp 1.0.0.1:853: i/o timeout
2022-09-26T17:01:16Z WARN dial tcp 8.8.4.4:853: i/o timeout
2022-09-26T17:32:13Z INFO planned periodic restart of DNS server
2022-09-26T17:32:13Z INFO downloading and building DNS block lists
2022-09-26T17:32:13Z INFO 0 hostnames blocked overall
2022-09-26T17:32:13Z INFO 0 IP addresses blocked overall
2022-09-26T17:32:13Z INFO 13 IP networks blocked overall
2022-09-26T17:32:13Z INFO starting DNS server
2022-09-26T17:32:13Z INFO DNS server listening on :53
2022-09-27T17:32:13Z INFO planned periodic restart of DNS server
2022-09-27T17:32:13Z INFO downloading and building DNS block lists
2022-09-27T17:32:13Z INFO 0 hostnames blocked overall
2022-09-27T17:32:13Z INFO 0 IP addresses blocked overall
2022-09-27T17:32:13Z INFO 13 IP networks blocked overall
2022-09-27T17:32:13Z INFO starting DNS server
2022-09-27T17:32:13Z INFO DNS server listening on :53
2022-09-28T17:32:13Z INFO planned periodic restart of DNS server
2022-09-28T17:32:13Z INFO downloading and building DNS block lists
2022-09-28T17:32:13Z INFO 0 hostnames blocked overall
2022-09-28T17:32:13Z INFO 0 IP addresses blocked overall
2022-09-28T17:32:13Z INFO 13 IP networks blocked overall
2022-09-28T17:32:13Z INFO starting DNS server
2022-09-28T17:32:13Z INFO DNS server listening on :53
2022-09-29T17:32:13Z INFO planned periodic restart of DNS server
2022-09-29T17:32:13Z INFO downloading and building DNS block lists
2022-09-29T17:32:13Z INFO 0 hostnames blocked overall
2022-09-29T17:32:13Z INFO 0 IP addresses blocked overall
2022-09-29T17:32:13Z INFO 13 IP networks blocked overall
2022-09-29T17:32:13Z INFO starting DNS server
2022-09-29T17:32:13Z INFO DNS server listening on :53

[darrencloutier]

Here's a grafana screenshot that goes along with these logs.

[qdm12]

I didn't test it yet, but 903b287471a9966d2111f0bb3e0d657024c1d1e0 should fix it, it now caches metrics across restarts (a bit of an ugly Go piece of code, but I couldn't think of something else or downside in doing so). Feel free to try the v2.0.0-beta image again, otherwise I'll run it someday soon and close it if resolved. Thanks 👍 !!