batpred
batpred copied to clipboard
springfall2008
Historical day warnings
Describe the bug I am running the standalone predbat add on. I am experiencing continual warnings for their 'historical day' data.
I get warnings every 5 minutes as predbat updates its plan. They are nearly all the same with minor changes in minutes of number of kWh. My current predbat log spans 10 hours and has 577 of these warnings.
I always used to have such warnings using the appdaemon–predbat add on. I thought that the standalone version of predbat add-on worked better with historical day data.
Since changing to the standalone version, I have removed the appdaemon-predbat add on and I have deleted all related files.
Expected behavior No historical day warnings?
Predbat version v8.1.0 with add-on v1.1.7 Running with original HTML predbat plan card.
Environment details
- Inverter and battery setup GE AC3.0 inverter with two 8.2 kWh batteries
- Standard HAOS installer on pi4
Screenshots Example: from predbat add-on log
2024-06-22 15:15:07.750721: Warn: Historical day 3 has 215 minutes of gap in the data, filled from 6.64 kWh to make new average 7.8 kWh (percent 85%)
2024-06-22 15:15:07.782964: Warn: Historical day 4 has 125 minutes of gap in the data, filled from 8.4 kWh to make new average 9.2 kWh (percent 91%)
2024-06-22 15:15:07.806988: Warn: Historical day 5 has 165 minutes of gap in the data, filled from 7.5 kWh to make new average 8.47 kWh (percent 89%)
2024-06-22 15:15:07.856336: Warn: Historical day 6 has 180 minutes of gap in the data, filled from 7.52 kWh to make new average 8.6 kWh (percent 88%)
2024-06-22 15:15:07.900069: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.48 kWh to make new average 11.89 kWh (percent 97%)
2024-06-22 15:15:07.936665: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.67 kWh to make new average 10.01 kWh (percent 97%)
2024-06-22 15:15:16.522980: Info: record_status Idle
2024-06-22 15:20:07.670479: Warn: Historical day 3 has 210 minutes of gap in the data, filled from 6.66 kWh to make new average 7.8 kWh (percent 85%)
2024-06-22 15:20:07.696102: Warn: Historical day 4 has 130 minutes of gap in the data, filled from 8.4 kWh to make new average 9.23 kWh (percent 91%)
2024-06-22 15:20:07.719013: Warn: Historical day 5 has 160 minutes of gap in the data, filled from 7.5 kWh to make new average 8.44 kWh (percent 89%)
2024-06-22 15:20:07.768096: Warn: Historical day 6 has 185 minutes of gap in the data, filled from 7.51 kWh to make new average 8.62 kWh (percent 87%)
2024-06-22 15:20:07.825272: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.49 kWh to make new average 11.9 kWh (percent 97%)
2024-06-22 15:20:07.862120: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.64 kWh to make new average 9.99 kWh (percent 97%)
2024-06-22 15:22:29.002140: Info: record_status Idle
2024-06-22 15:25:09.931290: Warn: Historical day 3 has 210 minutes of gap in the data, filled from 6.68 kWh to make new average 7.82 kWh (percent 85%)
2024-06-22 15:25:09.957284: Warn: Historical day 4 has 130 minutes of gap in the data, filled from 8.4 kWh to make new average 9.24 kWh (percent 91%)
2024-06-22 15:25:09.980002: Warn: Historical day 5 has 155 minutes of gap in the data, filled from 7.5 kWh to make new average 8.4 kWh (percent 89%)
2024-06-22 15:25:10.026569: Warn: Historical day 6 has 190 minutes of gap in the data, filled from 7.5 kWh to make new average 8.64 kWh (percent 87%)
2024-06-22 15:25:10.077235: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.5 kWh to make new average 11.91 kWh (percent 97%)
2024-06-22 15:25:10.116242: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.61 kWh to make new average 9.96 kWh (percent 97%)
2024-06-22 15:25:19.076676: Info: record_status Idle
Log file Current predbat.log attached: predbat.log
The warnings are regarding periods of zero energy usage, which its assuming is an error of recording. Maybe if you are a low user it could be possibly correct as the increments are only in 0.1 kWh I think.
The warnings are regarding periods of zero energy usage, which its assuming is an error of recording. Maybe if you are a low user it could be possibly correct as the increments are only in 0.1 kWh I think.
Is there any way we can confirm this is definitely what's happening and what time periods it is considered to be a gap (turn debug mode on?)
Looking at my house load it is plausible, the long period on the 16th of negative house load is because my import/export meters were not enabled last weekend (givenergy support **** up)
Need to put some logic in my template house load sensor to stop the negative/zero values if one of the underlying sensors is unavailable
2024-06-23 21:35:03.014749 INFO pred_bat: Warn: Historical day 3 has 10 minutes of gap in the data, filled from 46.1 kWh to make new average 46.43 kWh (percent 99%)
2024-06-23 21:35:03.022149 INFO pred_bat: Warn: Historical day 4 has 55 minutes of gap in the data, filled from 50.77 kWh to make new average 52.79 kWh (percent 96%)
2024-06-23 21:35:03.028686 INFO pred_bat: Warn: Historical day 5 has 5 minutes of gap in the data, filled from 21.02 kWh to make new average 21.1 kWh (percent 100%)
2024-06-23 21:35:03.038620 INFO pred_bat: Warn: Historical day 7 has 20 minutes of gap in the data, filled from 46.56 kWh to make new average 47.22 kWh (percent 99%)
2024-06-23 21:35:03.045531 INFO pred_bat: Warn: Historical day 8 has 210 minutes of gap in the data, filled from 68.81 kWh to make new average 80.56 kWh (percent 85%)
I can add something to the documentation if negative/zero load does cause this message
I have switched on 'debug enable' for 30 minutes from 10:30 pm to 11:00 pm on 23rd June. The debug enable seems to have stripped out many of the historical day warnings from predbat.log. If this shows up for periods of zero energy usage, I have been getting 6 warnings every 5 minutes like clockwork. That is 1728 per day!
This is a clean install of the standalone predbat and I have deleted anything associated with appdaemon.
I have attached my predbat.log, predbat add on log and one of the debug files from this evening. predbat add on log.txt predbat.log predbat_debug_22_55_00.yaml.txt
Predbat is running smoothly but the logs are getting swamped by the warnings, especially the predbat add on log. If there is nothing untoward with these warnings, should they be suppressed?
This is the relevant section from my apps.yaml file:
days_previous:
- 2
- 3
- 4
- 5
- 6
- 7
- 8
days_previous_weight:
- 1
Rob
I'm also seeing warnings which I assume are the cause of Predbat thinking usage is pretty much zero for many hours overnight. The absolute minimum draw even when the house is empty is 800-900W overnight so they are not correct that there is zero draw. They also often decrease in duration and don't seem to be consistent from one day to the next:
` 2024-06-24 13:35:02.806266 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.39 kWh to make new average 66.47 kWh (percent 88%) 2024-06-24 13:35:02.810929 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 58.29 kWh to make new average 59.75 kWh (percent 98%) 2024-06-24 13:40:02.816092 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.54 kWh to make new average 66.64 kWh (percent 88%) 2024-06-24 13:40:02.820667 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 58.1 kWh to make new average 59.55 kWh (percent 98%) 2024-06-24 13:45:02.770606 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.75 kWh to make new average 66.88 kWh (percent 88%) 2024-06-24 13:45:02.774912 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.76 kWh to make new average 59.2 kWh (percent 98%) 2024-06-24 13:50:02.830270 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.71 kWh to make new average 66.83 kWh (percent 88%) 2024-06-24 13:50:02.835167 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.45 kWh to make new average 58.88 kWh (percent 98%) 2024-06-24 13:55:02.773380 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.94 kWh to make new average 67.1 kWh (percent 88%) 2024-06-24 13:55:02.777915 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.13 kWh to make new average 58.55 kWh (percent 98%)
2024-06-25 13:35:02.231452 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 51.64 kWh to make new average 53.5 kWh (percent 97%) 2024-06-25 13:35:02.234065 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.39 kWh to make new average 66.47 kWh (percent 88%) 2024-06-25 13:40:02.212376 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 51.18 kWh to make new average 53.02 kWh (percent 97%) 2024-06-25 13:40:02.215064 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.54 kWh to make new average 66.64 kWh (percent 88%) 2024-06-25 13:45:02.735502 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 50.69 kWh to make new average 52.51 kWh (percent 97%) 2024-06-25 13:45:02.738258 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.75 kWh to make new average 66.88 kWh (percent 88%) 2024-06-25 13:50:02.741152 INFO pred_bat: WARN: Historical day 3 has 55 minutes of gap in the data, filled from 50.43 kWh to make new average 52.43 kWh (percent 96%) 2024-06-25 13:50:02.743978 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.71 kWh to make new average 66.83 kWh (percent 88%) 2024-06-25 13:55:02.718396 INFO pred_bat: WARN: Historical day 3 has 60 minutes of gap in the data, filled from 49.92 kWh to make new average 52.09 kWh (percent 96%) 2024-06-25 13:55:02.721249 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.94 kWh to make new average 67.1 kWh (percent 88%) `
Is there an additional debug level that displays the actual times that are missing and which sensor it thinks is blank?
Have you checked the load entity from GivTCP for the dates affected to rule that in/out?
I suspect you may be seeing something like mine below, where the load energy value doesn't seem to increment correctly. This is either a GivTCP or inverter firmware bug in my view, I would get lots of missing data warnings as zero usage in an hour isn't realistic.
To fix, I ended up creating 2 helpers. The first integrates the GivTCP power entity, I then feed this into a utility meter. This seems to work/increment better than the GivTCP provided entity. I then set this in apps.yaml.
Alternatively there is a bug
Finally came back to this and spent a while debugging and found out that I had used the inverter AC energy sensor to provide the load instead of the calculated house load which explains a lot.
load_today:
#- sensor.solaredgemodbus_i1_ac_energy
# MJI 26Jul24 Need total house load, not inverter AC values (doh!)
- sensor.calc_total_loads_energy_today
Changed it to the correct helper (which calculates the InverterAC + GridImport - GridExport - BattCharge) and all the warnings about historical gaps are gone and my charge/discharge plans are now a lot more like I'd been expecting!
My historical day warnings in my predbat add-on log have continued every 5 minutes and I must admit I have just ignored them. e.g.
2024-07-26 16:55:07.537579: Warn: Historical day 2 has 60 minutes of gap in the data, filled from 5.79 kWh to make new average 6.04 kWh (percent 96%) 2024-07-26 16:55:07.544892: Warn: Historical day 3 has 305 minutes of gap in the data, filled from 4.95 kWh to make new average 6.28 kWh (percent 79%) 2024-07-26 16:55:07.576204: Warn: Historical day 4 has 180 minutes of gap in the data, filled from 4.96 kWh to make new average 5.67 kWh (percent 88%) 2024-07-26 16:55:07.608204: Warn: Historical day 5 has 25 minutes of gap in the data, filled from 7.4 kWh to make new average 7.53 kWh (percent 98%) 2024-07-26 16:55:07.623134: Warn: Historical day 6 has 360 minutes of gap in the data, filled from 5.7 kWh to make new average 7.6 kWh (percent 75%) 2024-07-26 16:55:07.704461: Warn: Historical day 7 has 195 minutes of gap in the data, filled from 5.6 kWh to make new average 6.48 kWh (percent 86%) 2024-07-26 16:55:16.304369: Info: record_status Idle
In the apps.yaml documentation, it states:
Load filtering By default if Predbat sees a gap in the historical load data it will fill it with average data. This is to help in the cases of small amounts of lost data. For entire lost days you should change days_previous to point to different days(s) or include 3 or more days and if you set switch.predbat_load_filter_modal to true, the lowest day's historical load will be discarded.
load_filter_threshold - Sets the number of minutes of zero load data to be considered a gap (that's filled with average data), the default is 30. To disable, set it to 1440.
Looking back at the times in the add-on log, I have commented out the default '30' minutes and have replaced it with '480' minutes, equivalent to 8 hours. I don't know whether this is wise but I no longer see the warnings. Looking at the Givenergy power graph, I never get any time with zero load, the minimum power typically falling to 100-200 W. I can easily change the app.yaml code back to the default of 30.
Rob