ha-hildebrandglow-dcc
ha-hildebrandglow-dcc copied to clipboard
HandyHat
Data appears incomplete when compared to other DCC fed sources.
Describe the bug Read out is always less than what actually using from iHD, and Loop Energy DCC driven system.
I think this is the same as issue #292 where the accumulating cost appears to go down/reset at approximately 1.30am, resulting in the days total cost being lower than it otherwise should be.
I think this is the same as issue #292 where the accumulating cost appears to go down/reset at approximately 1.30am, resulting in the days total cost being lower than it otherwise should be.
Homes assistant is reporting 18.1kWh usage yesterday Loop energy and iHD are reporting 26.3kWh that's a significant difference, and my EV charger is reporting 17kWh usage yesterday just to charge my car.
Do you charge your car overnight? If HomeAssistant is losing/resetting it's energy accumulation around 1.30am, then perhaps that would account for the significant discrepancy lost while charging your car. Presumably if you don't charge your car (or avoid using energy 12am-2am) then the discrepancy would be much smaller.
I think the developer is investigating (hopefully).
Is the integration losing/resetting the kWh consumed? Or is it just the cost of the kWh consumed?
Looking at my own history charts & those posted by others in issue #292 , it seems to be the kWh consumed anywhere between 12am & around 8am (or possibly all the way up to 12pm) are lost/reset. So if you are charging your car (or other significant consumption) overnight, then that would be why you see a significant discrepancy.
Hopefully @HandyHat is on the case.
I am seeing this too. The new algorithm is considerably under reporting compared with the bright app. I think the issue may be the new algorithm does not “catch up” if there is a missed reading. The original algorithm caught up so was accurate, albeit putting the consumption in the wrong time slot.
Whether this issue is linked to the resetting of data of not, my DCC data seems to begin the day increasing normally until between 0102-0104 hours and then stops with various issues (readings going up/down and remaining flat) until resuming at a later point in the day when it fails to catch up with the DCC recorded totals.
I am also using a template sensor to sum the majority of my home's usage through kasa and tapo wifi plugs combined with powercalc integration to do my lights and appliances with stable/constant usage. When I map both and compare with bright app readings both come up short but the DCC via HA is dreadful. Bright app for the 6th is 8.98, 7th 6.804, 8th 6.735 The system has gone until 1733 hours before starting to increase normally but never catch up.
Looking at my own history charts & those posted by others in issue #292 , it seems to be the kWh consumed anywhere between 12am & around 8am (or possibly all the way up to 12pm) are lost/reset. So if you are charging your car (or other significant consumption) overnight, then that would be why you see a significant discrepancy.
Hopefully @HandyHat is on the case.
I have similar issues. Big gaps in the data and the daily totals are way out compared to actual usage.
This was yesterday, bearing in mind we have storage heaters and most of our energy use is between 00.30 and 07.30!
Also seeing large gaps in my data. Last time I had a full day of data was 23rd Jan. Mostly seems to be missing between 2am to around 8am ±1hr. Today is particularly bad as I only have 4 data points so far (00:00, 01:00, 12:00 and 18:00).
I too have gaps in the data, just as @Markbez graph above and others.
In my case at least, during the day I also have gaps in the data on the phone app as well, so I think for me it is possibly a server side issue (I emailed Hildebrand support yesterday to ask if it was a widespread issue, awaiting their reply - although I appreciate API access to their servers might not be a priority for them).
Interestingly, sometime later the missing data in the app seems to get backfilled, but this doesn't also backfill into Home Assistant.
I emailed Hildebrand support yesterday to ask if it was a widespread issue
I had a detailed and friendly reply, it seems that the data gaps are a capacity issue due to the popularity of the service, they are upgrading the infrastructure as soon as they can.
Looking at my own history charts & those posted by others in issue #292 , it seems to be the kWh consumed anywhere between 12am & around 8am (or possibly all the way up to 12pm) are lost/reset. So if you are charging your car (or other significant consumption) overnight, then that would be why you see a significant discrepancy. Hopefully @HandyHat is on the case.
I have similar issues. Big gaps in the data and the daily totals are way out compared to actual usage.
This was yesterday, bearing in mind we have storage heaters and most of our energy use is between 00.30 and 07.30!
My graphs look just like yours and my HA is only showing about 15~20% of my genuine consumption. This situation is getting worse every day and yesterday I kept watching the Bright App to see what happened. At 09:00, the App showed the first two half hour periods. It did not change state until 18:30 when it started adding in the data for the early hours. HA mirrored this with a flatline from 01:00 to 18:00 and then slowly increasing each half hour to midnight. At midnight, the HA integration did it's reset but the Bright App was only showing data to 08:00, around 16hours of lag. As I write this, the Bright App is showing the first two half hours of today, but yesterday is incomplete with data from midnight to 11:30. I am still waiting for 12hours worth of data from yesterday. I know the data is there in the DCC as my suppliers website shows it.
Similar issues here. It's making the integration very unhelpful. Note that both the Bright app and my supplier are showing the right data for yesterday now, but HA isn't and the Bright app is only showing a gas figure for 1am at present (4:30 pm).
Similar issue here too, the integration only seem to work if Bright App/DCC feed is timely. For the past week or so my feed is no where near real-time, it seems that it catches up todays readings in the early hours of the next day, and the integration does not do any catch up and I always get incorrect figures now.
exactly the same issue I am experiencing too. Data is incomplete between the exact time frames as above
Just been directed to this bug from the main discussion. I unfortunately am seeing the exact same issue. The Bright App and my SMETS2 meter are correct but what's reported in HA is missing data: https://github.com/HandyHat/ha-hildebrandglow-dcc/discussions/310
Is there any additional debug logging or similar I can provide from my instance that may help solve it?
Thanks
I also contacted Hildebrand and they were talking about a couple of weeks to upgrade their infra, so this issue is likely to be around for a while. They did mention that if you have the CAD then you shouldn't have these issues so I can test that when mine arrives in a few days. It should connect directly with the meter.
Interesting, so is this an issue at Hildebrand with their infrastructure rather than with this integration?
I am keen to get my hands on one of their SMETS CAD's without display, but been on the waiting list for ages now and still no sign. Get the impression it's in high demand so when their new production run arrives it'll be sold out almost immediately.
Hey all, as correctly deduced by some of you, this issue lies with the DCC/Bright providing the data later than the 30 minute delay, causing Home Assistant to be further behind and miss out the end of the day.
This seems to have deteriorated as 2023 progressed, and so I have contacted Hildebrand to see if they have any pointers.
I do have an idea on how to resolve this, but I'm not 100% sure it'll work - watch this space!
I am also experiencing something similar. Difference in readings between Bright and HA over longer periods. See below at images.
PS also notice Bright treats costs as 16bit signed number meaning 32.768 p/kWhr is max and higher than that as with today's exorbitant prices means negative Values. I'll use a template to correct for this 65.536 - (value).
I had an update this week from Jane at Hildebrand "This implies your software is making a different api call to get the consumption data than bright does. If someone tells us exactly what call is not working compared to bright we will have a look. Ideally please raise a ticket on this specific issue by emailing [email protected] The reality is if you want realtime up to date data buy the glowmarkt IHD and get a local mqtt client - everything else is something of a fudge."
Its a free service so we should not really have high expectations of them but whoever wrote the API probably needs to raise a support ticket and be patient.
That doesn’t explain why their own app isn’t updating until over 24 hours later! Or maybe it does if they are using the same api!
Sent from my iPhone
On 25 Feb 2023, at 06:15, daknightuk @.***> wrote:
I had an update this week from Jane at Hildebrand "This implies your software is making a different api call to get the consumption data than bright does. If someone tells us exactly what call is not working compared to bright we will have a look. Ideally please raise a ticket on this specific issue by emailing @.@.> The reality is if you want realtime up to date data buy the glowmarkt IHD and get a local mqtt client - everything else is something of a fudge."
Its a free service so we should not really have high expectations of them but whoever wrote the API probably needs to raise a support ticket and be patient.
— Reply to this email directly, view it on GitHubhttps://github.com/HandyHat/ha-hildebrandglow-dcc/issues/305#issuecomment-1445008150, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALI4GFRYASSCD43BXRX576DWZGPO5ANCNFSM6AAAAAAURVHF4U. You are receiving this because you were mentioned.Message ID: @.***>
So it looks that this integration is useless now... Is it only way now to get actual data in real time is using Hildebrand IHD?
So it looks that this integration is useless now... Is it only way now to get actual data in real time is using Hildebrand IHD?
Honestly, this was my thought too. Not sure if Hildebrand are essentially deprecating the HA/community use of the API.
Possibly related: I was aware people in the past could request for Cloud MQTT access for HA integration, I asked this question (ironically a couple of day before the troubles started) and got this response from Jane Wilson kinda sidestepping the request saying that this could only (now?) be done with their CAD:
As you realise, you will need our CAD for MQTT as it is intended to be used - MQTT is a publish, in near real-time, of data collected on the HAN from your meters by our devices. It is formatted into some json objects and then published for the user to consume.
There is no (and cannot be an) equivalent of the half hour data for MQTT. If you absolutely need the 30 minute data in MQTT you will need to code against our published API and then inject it into whatever broker you wish.
API access is automatic; you use your Bright credentials. Further information here : Repository URL - https://bitbucket.org/ijosh/brightglowmarkt/src/master/
The Bright Application ID is b0f1b774-a586-4f72-9edd-27ead8aa7a8d
This is our Display CAD: https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers - that comes with local MQTT and cloud MQTT on request.
All said, this is kinda fair enough honestly I was surprised there was any solution to getting detailed (near) real-time access to your own energy data, let alone a free one. While personally I think the price tag of the CAD/IHD is a little steep @ £70 you will end up with a truly local solution and that nicely fit the sprit of Home Assistant, so yeah, I'll probably end up going this route.
would think about the smet2 stick that they have been talking about for a while but never seems to have appeared
Yeah local connection sounds really good, but £70 is a big price tag. I have Chameleon IHD from Bulb that used to work with ST. Now that integration is totally messed up as well... I suppose since there is no other options to read our meters I might go for CAD/IHD as well.
I thought it was something I did! I have the same issue :/
May also go via the CAD/IHD route. £70 though..? hmm. What realtime data will that provide? Obviously consumption as you go through the day would be good. But, will it give you realtime instance usage? Anyone got one that can provide some info here?
Previously I was using a Shelly EM (with a CT clamp on the live feed at the meter). But after I got a smart meter fitted I switched to this - I thought that the readings would be more accurate if they were read from the meter, and I could also do away with an extra device (the Shelly).
I understand completely that this integration and the API from Hildebrand are all in "best endeavors" and good will, so no complaints from me - but after trying both I've now decided to go back to the Shelly. I was still using it anyway to measure "solar power back into the grid" - a function which Hildebrand say is still in the pipeline as a function for the CAD, if I were to go that route.
The Shelly + clamp were approximately the same cost that the CAD would be, provides near-realtime readings which I think are close enough to the readings from the meter to be good enough for me. It has MQTT and a web interface and integrates into HA. Minor point, but it is also not tied to the meter so can be taken if I move house. It does require 240V power (for power and reference) and the clamp might be tight to fit in some meter installations. It won't suit everyone.
YMMV, but the Shelly works well (enough) for me. Best of luck to this integration and to Hildebrand though.
Apologies for straying from a bug report, but it might help others here to have another option to consider.
This is disappointing as it seemed like a good way to get my gas usage - I can get my electricity from my inverter. I'll have to look at other options instead.