batpred icon indicating copy to clipboard operation
batpred copied to clipboard

Load Issues since v8.26

Open markbw999 opened this issue 2 months ago • 45 comments

Cut and paste from the Facebook group...

"[Trefor Southwell], something strange is happening with my predbat instance since v8.26 yesterday. I first noticed it because I use Current Load (predbat.load_energy_h0) as an automation trigger and it dropped to zero after the update. I also looked at some of the other calculated entities and Predicted Load jumped out as anomalous (see first screenshot). I suspect it is related to warning messages in the log (see second screenshot). I haven't noticed these before yesterday and haven't knowingly changed anything. Finally, and this predates yesterday's update, my cost savings seem to have dramatically changed pattern (see third screenshot). Any suggestions? Thanks."

Log and debug attached as requested.

predbat.log predbat_debug.txt

markbw999 avatar Oct 22 '25 08:10 markbw999

I'm also seeing this issue, with the same warning in the log: Warn: Historical day 7 has no data, unable to fill gaps normally using nominal 24kWh - you should fix your system!

Rolling back to 8.25.14 has temporarily resolved the issue for me.

8.25.14: Image

8.26.1: Image

AlexCPU avatar Oct 22 '25 20:10 AlexCPU

This appears to be EMS related, can you share your apps.yaml?

springfall2008 avatar Oct 23 '25 22:10 springfall2008

apps.yaml

Mine's a complicated mess of ac-coupled solaredge for PV and a separate Victron Multiplus for energy storage.

AlexCPU avatar Oct 24 '25 06:10 AlexCPU

Here you go...

apps.yaml

markbw999 avatar Oct 24 '25 07:10 markbw999

Are you using GivEnergy kit? If so I think the problem is the GivTCP addon. The load energy today sensor is broken (sensor.givtcp_fd23xxxxxxxxx_load_energy_today_kwh) and is not reporting or resetting. I know that the one I use to forecast load etc.

Timeflies1980 avatar Oct 31 '25 08:10 Timeflies1980

Here you go...

apps.yaml

I notice in your apps.yaml you have:


load_today:
    - sensor.ge_inverter_{geseriale}_consumption_power

this looks like an instantaneous power sensor, not the daily increasing energy sensor. Are you sure this is correct?

And as OP mentions, there is a bug in givtcp 3.4.1 that is not resetting the load today sensor correctly. See givtcp repository for details

gcoan avatar Oct 31 '25 08:10 gcoan

Are you using GivEnergy kit? If so I think the problem is the GivTCP addon. The load energy today sensor is broken (sensor.givtcp_fd23xxxxxxxxx_load_energy_today_kwh) and is not reporting or resetting. I know that the one I use to forecast load etc.

I am using GE kit but I have an EMS that requires the GE Cloud integration as GivTCP doesn't fully support EMS yet.

markbw999 avatar Oct 31 '25 08:10 markbw999

Here you go... apps.yaml

I notice in your apps.yaml you have:


load_today:
    - sensor.ge_inverter_{geseriale}_consumption_power

this looks like an instantaneous power sensor, not the daily increasing energy sensor. Are you sure this is correct?

And as OP mentions, there is a bug in givtcp 3.4.1 that is not resetting the load today sensor correctly. See givtcp repository for details

It is the power sensor ; I was using the load equivalent but was getting strange outcomes (e.g. it would drop during the day rather than continuously rising). I reported it as a problem and Trefor advised at the time to use the power sensor as predbat will apply an integral to convert to energy.

markbw999 avatar Oct 31 '25 08:10 markbw999

It is the power sensor ; I was using the load equivalent but was getting strange outcomes (e.g. it would drop during the day rather than continuously rising). I reported it as a problem and Trefor advised at the time to use the power sensor as predbat will apply an integral to convert to energy.

OK, you're in the wild west anyway if you're doing that as advised by Trefor!

What does your load today energy sensor look like?

An alternative approach is to create a load today sensor manually from import, export, battery charge, discharge, PV, etc. There is an example in the documentation. I use this because with two inverters that share the house load the individual load sensors are meaningless. This might be required for EMS as well?

gcoan avatar Oct 31 '25 08:10 gcoan

Image Like this - as I said, it shouldn't dip during the day.

I should also point out I'm on the latest version of GivTCP (even though predbat isn't using it). Ihave reverted to predbat v8.25.14 and everything is working fine. IMHO it was something intrduced in v8.16 that is causing these issues.

markbw999 avatar Oct 31 '25 09:10 markbw999

I don't have any GivEnergy kit in my setup and see the same errors in my logs. As per Mark, without any changes to the config, swapping back to 8.25.14 everything works fine. I'm trawling through the git compare between versions trying to guess at what might be the problem, but don't know the codebase enough to guess.

AlexCPU avatar Oct 31 '25 09:10 AlexCPU

thanks guys, reverting back to an older predbat version does make it look like a recent predbat bug

gcoan avatar Oct 31 '25 09:10 gcoan

I'm going to watch this topic - I think I am seeing the same thing. My predbat logs show:

2025-10-31 12:15:01.835141: Warn: Historical day 7 has no data, unable to fill gaps normally using nominal 24kWh - you should fix your system!

However so far as I can see, the sensor.givtcp_xxxxx_load_energy_today_kwh sensor looks fine

Rolling back to 8.25.14 has not so far helped

dbrb2 avatar Oct 31 '25 12:10 dbrb2

I've tried stripping my apps.yaml all the way back to just having the load_today sensor and inverter settings and I'm still seeing the problem.

I've turned on debug and diff'd the debug.yaml's, and whilst there's a significant difference in all the predictions (because it's not using any load data), the only thing of note that I can immediately spot, is some of the sensors are missing data, for example, predbat.best10_import_energy in 8.25.14 has 340 lines of data, whereas in 8.26.4 only has 29 lines of data.

Debug files: predbat_debug-broken-8.26.4.yaml predbat_debug-working-8.25.14.yaml

Not sure if this is relevant, but occasionally I'm getting the below error in 8.25.14. It only seems to show up when I've been messing with settings (like I have today trying to find anything here!), and if I do a full restart of predbat it doesn't reappear, even if left for days:

47947 | 2025-10-31 11:15:50.035692: Error: 'NoneType' object has no attribute 'split'
47946 | 2025-10-31 11:15:50.035629: Info: record_status Error: Exception raised 'NoneType' object has no attribute 'split'
47944 | AttributeError: 'NoneType' object has no attribute 'split'
47943 | ^^^^^^^^^ ^^^^^^
47942 | entity_base = entity_id .split(".")[0]
47941 | File "/config/inverter.py ", line 1528, in write_and_poll_option
47940 | self.write_and_poll_opt ion("inverter_mode", entity_id, new_inverter_mode)
47939 | File "/config/inverter.py ", line 1700, in adjust_inverter_mode
47938 | self.adjust_inverter_mo de(force_export)
47937 | File "/config/inverter.py ", line 1879, in adjust_force_export
47936 | inverter.adjust_force_e xport(False)
47935 | File "/config/execute.py" , line 611, in reset_inverter
47934 | self.reset_inverter()
47933 | File "/config/execute.py" , line 33, in execute_plan
47932 | ^^^^^^^^^^^^^^^^^^^
47931 | status, status_extra = self.execute_plan()
47930 | File "/config/predbat.py" , line 712, in update_pred
47929 | self.update_pred(schedu led=True)
47928 | File "/config/predbat.py" , line 1503, in run_time_loop
47927 | 2025-10-31 11:15:50.025981: Error: Traceback (most recent call last):
47926 | 2025-10-31 11:15:50.025156: Error: Exception raised 'NoneType' object has no attribute 'split'

AlexCPU avatar Oct 31 '25 12:10 AlexCPU

My error:

2025-10-31 12:15:01.835141: Warn: Historical day 7 has no data, unable to fill gaps normally using nominal 24kWh - you should fix your system!

Started to give a gap size of about 1300 minutes

It fell by 5 minutes every run until it reached:

Historical day 7 has 1120 minutes of gap in the data

At which point it stopped falling and had been sitting at that gap size ever since

dbrb2 avatar Nov 01 '25 08:11 dbrb2

Ah. This seems to be down to a known bug in givtcp which does indeed stop updating the sensors on occasion

dbrb2 avatar Nov 01 '25 18:11 dbrb2

Ah. This seems to be down to a known bug in givtcp which does indeed stop updating the sensors on occasion

That would figure for your problem where rolling back doesn't fix it. I don't use GE kit at all. So there's some other bug in 8.26 onwards.

AlexCPU avatar Nov 01 '25 18:11 AlexCPU

Ah. This seems to be down to a known bug in givtcp which does indeed stop updating the sensors on occasion

I have just upgraded to givtcp Dev 3.4.16 which might include a fix for the load sensor problem on givenergy kit

But agree, there does appear to be a predbat issue as well as non-givenergy users are having problems with the latest predbat release.

gcoan avatar Nov 01 '25 18:11 gcoan

Can you confirm if the problem relates to using a power sensor for energy or if it's wider than that? My load energy h0 works fine

springfall2008 avatar Nov 01 '25 21:11 springfall2008

I'm using a constantly increasing energy sensor for load_today.

Not sure if it's relevant, but it's showing in Wh, when the rest are in kWh:

Image

AlexCPU avatar Nov 01 '25 21:11 AlexCPU

Is it really in Wh, what does it look like in HA?

springfall2008 avatar Nov 01 '25 21:11 springfall2008

Yep, looks like Wh in HA too.
Image

AlexCPU avatar Nov 01 '25 21:11 AlexCPU

Looking a bit closer, my load_today Wh sensor is just an integral helper. If I give the raw underlying sensor to predbad, on 8.25.14 it behaves exactly the same as the integrated sensor, on 8.26 I'm getting this warning in the logs Warn: Historical day 7 has 1210 minutes of gap in the data, filled from 21.18 kWh to make new average 132.59 kWh (percent 16%)

These InDay charts are using the power sensor, rather than the integral energy sensor:

8.25 (this is correct - ignore the fact the prediction is way off, my load is fun): Image

8.26 (I don't know why it's not smooth, also it's clearly wrong): Image

AlexCPU avatar Nov 01 '25 22:11 AlexCPU

It's a +1 from me, I'm afraid. SolaX gen 4 hybrid here. Data for the four sensors looks pretty much ok. No real config changes. Let me know what info I can provide to help.

  load_today:
    - sensor.solax_house_energy_daily
  import_today:
    - sensor.solax_today_s_import_energy
  export_today:
    - sensor.solax_today_s_export_energy
  pv_today:
    - sensor.solax_today_s_solar_energy

M0LTE avatar Nov 03 '25 23:11 M0LTE

Just a bit of a deeper look, if I roll back to 8.25.14 it works, if I roll forwards to the next version 8.26.0 it's broken.

8.26.0 is labelled "Internal code improvements", I think you have a regression a few versions ago @springfall2008

M0LTE avatar Nov 03 '25 23:11 M0LTE

GPT5 reckons it's found it (apps/predbat/ha.py)

- from config import TIME_FORMAT_HA, TIMEOUT, TIME_FORMAT_HA_TZ
+ from config import TIME_FORMAT_HA, TIMEOUT, TIME_FORMAT_HA_TZ

- now = datetime.now(timezone.utc)
+ # Use local time so HA’s day boundaries match recorder/statistics
+ now = datetime.now()

- res = self.api_call("/api/history/period/{}".format(start.strftime(TIME_FORMAT_HA)),
-                     {"filter_entity_id": sensor, "end_time": end.strftime(TIME_FORMAT_HA)})
+ # Include timezone in start/end (HA accepts TZ-aware ISO strings)
+ fmt = TIME_FORMAT_HA_TZ if 'TIME_FORMAT_HA_TZ' in globals() else TIME_FORMAT_HA
+ res = self.api_call("/api/history/period/{}".format(start.strftime(fmt)),
+                     {"filter_entity_id": sensor, "end_time": end.strftime(fmt)})

M0LTE avatar Nov 03 '25 23:11 M0LTE

GPT5 reckons it's found it (apps/predbat/ha.py)

Personally I don't trust the AI engines to find such bugs

but worth a shot, you can just edit the ha.py file in the predbat addons folder, predbat will reload and start executing the new version

gcoan avatar Nov 04 '25 00:11 gcoan

No intention that the patch is applied directly, it is just a pointer which may or may not be helpful.

M0LTE avatar Nov 04 '25 00:11 M0LTE

I'm still stuck on 8.25.14

8.26.0 errors with "Warn: Historical day 7 has no data, unable to fill gaps normally using nominal 24kWh - you should fix your system!"

The bug is somewhere in this changeset

M0LTE avatar Nov 06 '25 20:11 M0LTE

@M0LTE if you can provide a logfile of predbat working on 8.25.14 and not on 8.26.0, that would be helpful to diagnosing the issue.

thanks

gcoan avatar Nov 06 '25 20:11 gcoan