Longterm statistics of utility meter summed up at midnight

[tzwickl]

The problem

I discovered a weird bug that at midnight when the daily utility meter is resetting that the sum of the last period is added to the sum of the longterm statistics.

When looking at the adjust statistics functionality in the UI I can see that there is a consumption of 1.88kWh at midnight which is exactly the amount of energy that was consumed in the last period:

Strangely the utility meter itself looks normal and is increasing all day until it is reset at midnight:

When looking in the DB at statistics I can see that at midnight the state of the utility meter changes from 1.85kWh to 0.239kWh and the sum of the sensor increases by 1.88kWh which should not happen right?

When looking in the DB before I upgraded to version 2023.5.3 it all looks good and as expected:

This bug seems to happen for all daily utility meters I have and is now messing up my energy dashboard as the consumption is way too high for all my devices.

What version of Home Assistant Core has the issue?

2023.5.3

What was the last working version of Home Assistant Core?

2023.4.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[functionpointer]

Same issue here, but slightly different parameters.

My entity is set as total_increasing. The values never go down. The underlying sensor does become unavailable during the night, but it never sends a bad value. It goes from unavailable straight to the same value as the evening before, and further updates only increase.

Still, when the first value comes the energy dashboard shows the full sum of the day before.

Weird thing is i can't find the problem in the db: statistics table: short_term_statistics table: underlying data: I have fetched the underlying data via API and verified there is no bad value anywhere. Literally 4.9 -> unavailable -> 4.9.

The data comes from MQTT. I am running HomeAssistant commit 5821a563c7e2002bc7d4c67b165d389ba7e6adfa, which is from the dev branch and newer than 2023.5.3. Also the bad data doesn't show up in the statistics tabs of the developer tools, so the "Adjust sum" tool can't help me.

[functionpointer]

Updated to commit https://github.com/home-assistant/core/commit/53e456f4530dd7d4b1722b2a5f39fda1f3bb2452, problem is gone. Both retroactively and for new days.

[joostlek]

Is this still an issue?

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.