Usage reporting as negative at the beginning of the day

[cegan09]

I was unable to find any other reports of this through search, apologies if I missed something.

I'm seeing most of my individual circuit sensors reporting a huge negative usage during the first hour of the day. It seems like what is happening is that the sensor is suddenly reporting some amount of the previous day's usage as generation for the first hour of the next day, and then the rest of the day that circuit is acting like it's trying to apply usage against that big generation.

I have a vue2 with 13 individual circuit sensors installed. I have no solar or any other kind of generation on site, I am 100% consumption. The emporia app does not have this issue, everything reports correctly. So it seems to be a discrepancy between the two.

I installed the vue2 early October, and it worked fine through October 13th. October 14th the day starts with the large generation. I do notice that on October 13th the 10am-11am block is missing, just nothing reported. It's like that day it shifted everything an hour and now it's still off.

I have reloaded the integration, I have not tried removing and re-adding it yet. I didn't know what diagnostic steps I can do to help figure out what happened. Let me know what I can do, if anything.

[krazos]

I'm having the same (or at least a similar) issue, which confuses long-term statistics and the Energy dashboard:

[magico13]

Since those are in the same hour the energy dashboard should be basically ignoring it. Can you show me a picture of how it's treating it? Maybe with solar it's different.

It's a side effect of me detecting midnight and resetting to zero, then the API giving data that seems to ignore that the day changed, then later the API giving appropriate data. We pull from the API every fifteen minutes, which is why it jumps around 12:05 and then drops around 12:20. I will try to prioritize correcting/ignoring the bad API data, I assumed that the jump and drop would negate in the statistics since they do for energy consumption.

[krazos]

Thanks for the quick response. This morning, the solar production graph in my energy dashboard was showing negative total solar production of approx. -3.5 kWh. I already fixed today's data anomaly in my long-term statistics using Developer Tools, but I will try to grab a screenshot tomorrow morning.

[krazos]

So, here's what I'm seeing.

I neglected to mention that I am using a template sensor to combine the readings from both phases of the 240V breaker that is connected to my solar inverter. (Emporia's supplemental installation guide for solar recommends installing a 50A CT on each of the leads coming in from the solar inverter to the breaker rather than relying on a multiplier.)

I will try instead adding the two separate phase readings to the energy dashboard instead of the calculated template sensor and check back tomorrow to see if that has any impact on the negative reading in the dashboard.

[cegan09]

This is what I'm seeing. The energy page shows this massive negative usage (generation) right in the 12-1am block. I do not have solar or any other way to generate electricity. Looking at the plot for individual circuits they seem right, meaning they ramp up to midnight then reset to zero.

It's looking like there might be a disconnect somewhere on where the reset point is. Meaning whatever is deciding where the data is pulled to show on the energy plots is an hour or more off, so it's capturing that drop to zero and registering it as a generation, instead of the reset point.

[magico13]

@krazos the template sensor might be state_class total and the last_reset time isn't being set so it's all one "cycle". If that value only goes up until it resets you might be able to set it to state_class total_increasing and have it recognize the new cycle automatically. Or if the data fluctuates (since the API isn't always consistent) then you might need to set the reset time so it treats the drop as a new cycle instead of a big negative.

@cegan09 can you share what timezone you have set in the Emporia app? The reset is determined from the timezone set in the Emporia app for the device, I'm detecting midnight and then resetting to zero and setting the last_reset time to the new midnight. There actually should be a message in the logs for that, it should look something like below. My timezone is America/New_York which is currently UTC-4, so my midnight is 04:00 UTC. The graph you show seems to line up with the drop happening just after midnight, but maybe the new reset time isn't being set correctly or something. I'm curious if that logged message has the right values in it.

2022-10-31 04:00:58.002 WARNING (MainThread) [custom_components.emporia_vue] Midnight happened recently for id 86116-1,2,3-1D! Timestamp is 2022-10-31 00:00:48-04:00, midnight is 2022-10-31 00:00:00-04:00, previous reset was 2022-10-30 00:00:00-04:00

[cegan09]

I'm also on the east coast, my emporia app is set to eastern, but it's showing UTC -5.

Where can I find the logs? I'm not able to find anything in the logbook for the emporia devices. I'm not sure why they're not showing.

[magico13]

Logs should be at https://{home-assistant-url}:8123/config/logs, or under Settings -> System -> Logs. I recommend scrolling down to the bottom and loading the full logs, then look around midnight this morning to find the most recent one. That's assuming you're running the latest version of the integration, which is v0.8. I'm also on a relatively up to date Home Assistant build.

Eastern time is "normally" UTC-5 but we're still in DST for another week. I wonder if this issue will go away while we're out of DST and then come back in the spring when we go back to -4.

[krazos]

@krazos the template sensor might be state_class total and the last_reset time isn't being set so it's all one "cycle". If that value only goes up until it resets you might be able to set it to state_class total_increasing and have it recognize the new cycle automatically. Or if the data fluctuates (since the API isn't always consistent) then you might need to set the reset time so it treats the drop as a new cycle instead of a big negative.

Thanks for the suggestion; I'll give that a shot. I typically use total_increasing for my energy sensors, but I had selected total for this one to match the device class of the sensor entities created by the integration. Hopefully changing the device class to total_increasing will do the trick!

[magico13]

I had issues with making these total_increasing due to the API giving occasional bad values or some overall energy sensors being legitimately non-monotonic so that's why they all swapped to total with last_reset in the last update. I am still planning on trying to filter out the bad API responses but just be aware that they might mess with your statistics as well if you use total_increasing. Hopefully that's infrequent though... 🤞

[krazos]

Noted. I was stoked to discover this integration. Thanks for all of the work that you put into it!

[cegan09]

I was trying to just use the sidebar logbook, not the full system logs. My mistake.

This is the message I'm seeing. I just picked one device at random to copy, they all have the same times in the message. Midnight happened recently for id 161826-10-1D! Timestamp is 2022-10-31 00:00:27-04:00, midnight is 2022-10-31 00:00:00-04:00, previous reset was 2022-10-30 00:00:00-04:00

[cegan09]

I'm still seeing this happen. I've fully removed the emporia integration and reinstalled it, with the same outcome. I've restarted everything, updated everything I can, I still get the huge generation at midnight. the only thing I haven't tried is pulling the panel cover off and physically rebooting the vue. I may try that later, I've just been avoiding poking inside the buzz box when I can avoid it.

[magico13]

The message looks like I would expect. Can you tell me what you have the time zone set to in both Home Assistant and in the Emporia settings? Here's both of mine for reference, left is Home Assistant in the Settings->General menu and right is the Emporia app in the Device Settings. I don't really understand how the energy dashboard is treating that as a bunch of generation and not as the start of a new cycle.

[magico13]

Another thing to check would be the state data in the Developer Tools in Home Assistant. When I pull up my home_1,2,3_1d sensor it looks like this. The state_class should be total and the last reset should be midnight in your local time.

[cegan09]

Mine looks just like yours does. Which does point to "this is a me issue" and something I did elsewhere with home assistant is messing it up.

The only weird thing I can point to is that the day before this started happening, there is an hour missing in the plots. This was October 13th, and the 10-11am hour just isn't there. Ever since then I have the big spike at midnight that throws everything off. I will see if I can dig into the logs more and look at that specific day.

[cegan09]

So I do not know what happened, but just as suddenly as this popped up, it went away. On the 8th everything just went back to normal.

Not sure why, but it looks like things are ok again.

[MassimoL06]

Hello @cegan09 and @magico13. I am experiencing same problem. I am located in France.

In the log is have this

This is my time zone in HA and I set the same in Emporia.

I have the problem on all my Emporia sensors.

I use the python script from PyEmVue-master to get the data. If I launch it few minutes after Midnight (00:03 mins) I still have the values from previous day: You can see in the screen (in yellow) that the given date is one hour before real time !!! If I launch it 20minutes after Midnight (00:20 mins), the values are reset :

So weird !

[MassimoL06]

Hello Again, the issue doesn't occur everyday. Just the first midnight after a HA restart.