ha-hildebrandglow-dcc
ha-hildebrandglow-dcc copied to clipboard
HandyHat
1000s of DNS lookups to api.glowmarkt.com
I use Pi-hole (see https://pi-hole.net/) on my LAN to filter out excessive internet adverts and also as my DHCP & DNS lookup server.
Sometime after installing this integration, I noticed many 1000's of DNS lookups to api.glowmarkt.com in my Pi-hole dashboard. There are 30+ lookups every 1second.
To see these lookups, you would need to setup Pi-hole or some other network device to receive & forward DNS queries. I have attached a screenshot.
Surely 30+ DNS lookups every 1 second is unnecessary and presumably not good for LAN, ISP or Hildebrand servers!?
Home Assistant v2021.10.6 Hildebrand Glow (DCC) integration v0.5.0
*** Update***
My test system seems to be behaving, and making only 8 lookups every 2 minutes, which is what I'd expect.
Similar on live.
So for my system, that is 8 * 30 * 2 requests per hour.
Today we are at 15:00 so 15 * 8 *30 * 2 = 7200.
In my logs I see 9000 DNS calls, so not too far out.
Looking at the glowmarkt DNS entry, they have the TTL set to 5 minutes.
api.glowmarkt.com | IN | A | 154.51.148.100 | 300s | (00:05:00)
This seems low, but doubt the full reason for the issue.
I have rebooted my Raspberry Pi that runs HomeAssistant (in a docker container) and as far as I can see, there are still runs of DNS lookups in Pi-hole. It looks like every 2mins there are 24 DNS lookups in the space of 2seconds - see part of the Pi-hole report below.
2021-10-2716:15:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
-- | -- | -- | -- | -- | -- | --
2021-10-2716:15:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.2ms) | Blacklist
2021-10-2716:15:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.2ms) | Blacklist
2021-10-2716:15:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.2ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.3ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.2ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.1ms) | Blacklist
2021-10-2716:15:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:15:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:13:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:13:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:13:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:13:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms) | Blacklist
2021-10-2716:13:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:13:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.1ms) | Blacklist
2021-10-2716:13:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (1.7ms) | Blacklist
2021-10-2716:13:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.2ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (1.6ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (1.8ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (2.4ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.5ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (2.3ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.6ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (2.3ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.7ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (3.1ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.8ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (2.5ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.3ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (2.5ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (2.5ms) | Blacklist
2021-10-2716:13:06 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | NODATA (3.6ms) | Blacklist
2021-10-2716:13:06 | A | api.glowmarkt.com | rpi4b-eth0 | OK, answered bylocalhost#5335INSECURE | IP (4.4ms) | Blacklist
2021-10-2716:11:07 | AAAA | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | NODATA (0.1ms) | Blacklist
2021-10-2716:11:07 | A | api.glowmarkt.com | rpi4b-eth0 | OK(cache)INSECURE | IP (0.2ms)
According to my Pi-hole, in the 24hrs till now, there have been 15,749 DNS lookups of api.glowmarkt.com. That's about 13,000 hits more than the next highest permitted domain lookup which is alexa.amazon.co.uk So it is kinda 'orders of magnitude' higher than anything else.
Thanks for the info. A set of DNS lookups every 2 minutes is what I would expect. The code makes a call for:
- Pull latest usage data from DCC to Glow
- Pull latest cost data from DCC to Glow
- Get current usage
- Get current cost
- Get cumulative usage
- Get Tariff data.
It will do this for Electric and Gas - so 12 "Get" requests.
It seems the code creates a new session for each request, so this might be where the other 12 come from.
An improvement would be to use requests.Session(), but that will take time to figure out how the HA model deals with that.
Okay, it definitely seems to be 24 lookups every 2mins taking ~2secs (in the Pi-hole dashboard log). I'll monitor it for now, but it would be good to reduce the lookups to something sensible as it WAY outstrips everything else on my network. I'm not sure whether it has a perceptible impact on network performance.
I've researched this a little.
The integration uses the python requests package, which makes a DNS query per-request.
Using a requests.session() might fix it, so made a quick fix, this cuts down the number of calls to one per request (I think as the session is open, so SSL established). I’ll keep, testing this an make a PR in due course as it’s better.
I guess we need to determine how other integrations deal with the DNS lookup issue.
I have posted a question on this in the wider Home Assistant community, to see what others may suggest:
https://community.home-assistant.io/t/multiple-dns-queries/350765
@smithbill17 daft question, are you using the integration or did you just install it to have a look at it? it might be a 401 error loop if you havent setup the credentials?
@smithbill17 daft question, are you using the integration or did you just install it to have a look at it? it might be a 401 error loop if you havent setup the credentials?
Yes, I'm using it. It's logged in with appropriate credentials & receiving data okay (although it still seems to behave a little oddly just after midnight, but that's a separate issue @ColinRobbins has already looked at). I still get 24 DNS lookups every 2mins.
I’ve got the 24 lookups down to 12, but am waiting for a week before I issue a PR. I do want to check there is not 401 loop as @smithbill17 suggests. This is a possibility after 7 days of running, when the token expires.
I still get 24 DNS lookups every 2mins.
i think thats divided by 2 so only 12 lookups every 2mins as its looking up AAAA and A together (ipv6 and ipv4)
i see it from my servers time to time, it looks up the IPv6 addresses even though i dont use IPv6
BUT then divide 12 by 2 again (2 services gas + elec) thats 6 requests each
but that would be right as if you look into the raw api https://api.glowmarkt.com/api-docs/v0-1/resourcesys/#/
it needs to call
- catchup (refresh readings)
- tariff prices
- readings for the 30min window
- readings for the whole day
- resources (to check if you have randomly added extra smart meters, you never know with the magic software fairys)
- refrsh token
That’s is not quite how it works. I have listed the 6 calls the code makes in a post above. 6 for gas and 6 for electric.
Looks like after the network issue Glow had with their systems last week/weekend ( https://forum.glowmarkt.com/index.php?p=/discussion/comment/540#Comment_540 ), they just posted an update on their forum with the following:
UPDATE from the Glow team
Because some users without our CAD are calling the API much too frequently (it should be once per half hour) we have had to alter the way the call is dealt with and reject the excess traffic.
Most on this forum have our CAD so the point is moot but FYI, we've got a lot of Home Assistant users of MQTT. We tell people that MQTT isn't meant for half hourly data but they are using brokers that seem to make a lot of redundant calls.
We hope to get our usual more frequent updates of the DCC data in place soon but wanted to explain why we are currently on overnight only. Please bear with us and check back in a day or two. Thank you.
Checking my pi-hole, since yesterday I can see 11676 DNS queries (in less than 24 hours) which seems a bit excessive. Is there anything we can do with regards to that? Just asking as they are throttling the calls and I am not (and probably others) able to get info at all to HA.
Since 1st Feb, I seem to be only getting a single reading at 5-6am every 24hrs (which is obviously pretty useless). If we're still making 1000s of DNS queries every day, then surely the only way forward is to stop retrieving 6 data points every 2mins and instead retrieve the same data every 30mins?
@smithbill17 i just suggested this in another issue https://github.com/HandyHat/ha-hildebrandglow-dcc/issues/126#issuecomment-1032600786
@smithbill17 i just suggested this in another issue https://github.com/HandyHat/ha-hildebrandglow-dcc/issues/126#issuecomment-1032600786
Nice, I haven't noticed that issue. I've seen a scan_interval on sensor.py, I changed from 2 minutes to 10 to test and it seems to work.
@smithbill17 i just suggested this in another issue #126 (comment)
Nice, I haven't noticed that issue. I've seen a scan_interval on glow.py, I changed from 2 minutes to 10 to test and it seems to work.
my fix should help the issue #135
Hey all, thanks for your work here. I agree - we are currently making too many calls to the Hildebrand API. I have got some suggestions on how to reduce this impact, and would love your feedback in the discussion: https://github.com/HandyHat/ha-hildebrandglow-dcc/discussions/191
Just to comment - the DNS queries are highly unlikely to be going anywhere near Hildebrand. Your API requests is a different discussion, which you already have underway, but the DNS requests should be cached in numerous places:
- The local system - most OSes will have some level of DNS cache
- The local router - most routers will set themselves as the DNS server, and forward requests to your configured DNS providers (or the ISP configured providers) - allowing for local network name resolution.
- Your upstream DNS servers - I can pretty much guarantee that your DNS queries are unlikely to be making it past your ISP / Cloudflare / Google / OpenDNS servers. Those servers will almost definitely be caching for the configured TTL. if not longer.
- Cascading DNS servers across the internet - If you somehow do manage to get past your configured upstream, then their configured upstream servers are also likely to be caching records.
- The domain registrar - It's unlikely that Hildebrand hosts its own nameservers. Far more likely is that they've given this job to Cloudflare, or Fastly, or Godaddy, or some other big name that is designed to serve immense amounts of DNS.
So we should definitely be careful to make sure that we're not making excessive DNS queries, but rest assured we're not DDoSing Hildebrand with this integration :)
