batpred inday adjustment does not carry forward over midnight

Describe the bug This may be by design, but I think the design isn't quite right, or at the least it needs a new configuration option to enable what I think should happen...

I have an ASHP. Today has been very much colder than it has previously been so the heat pump has been working harder (higher flow temperature) and consuming more electricity than normal.

Predbat is not catching up with that extra load with actual considerably ahead of predicted load:

Expected behaviour Predbat should recognise the extra load and adjust the forecast accordingly.

Further investigation and identification of the problem In the logfile I see predbat recognises the extra load:

21:25:02.616094: Today’s actual load so far 23.65 kWh with 4.71 kWh Car/iBoost excluded and 4.3 kWh import ignored.|
21:25:02.616016: Today’s load divergence 130.4 % in-day adjustment 128.88 % damping 0.95x|

But as shown, it is not adjusting for this. Earlier today in the Cosy cheap rate 1-4pm period, Predbat initially planned to charge the batteries for the whole period, but afterwards I saw it had changed to freeze charging for part of the period, leaving the batteries not full.

The batteries have run out resulting in grid import (when Predbat could have charged more earlier), and more, the plan is for very little charging in the Cosy cheap periods tonight and tomorrow morning so the batteries will run out again overnight as it continues to be cold:

I looked at inday adjustment and found that this was set to the default 0.95, so predbat is dampening down the extra load:

So to try to adjust the plan I set inday adjustment metric to an artificially high 1.5:

And the plan looks almost identical:

Looking more closely I see that the load TODAY up to midnight is increased over the projected load when inday metric was 0.95, but all the load TOMORROW is unchanged. i.e. predbat is adjusting today's load based on the metric adjustment, but at midnight that adjustment stops and load is unchanged after that.

Now this might be by design and the name of the metric, "inday adjustment" but for those of us with electric heating (ASHP) it's completely wrong. If its cold today and my house load is high as a result of the extra load then that extra load will continue through the night and Predbat should recognise and plan for that.

So either a bug, or a FR with a new option to allow inday adjustment to continue and scale house load past midnight

Predbat version

8.25.14

Environment details 2x Gen 1 hybrid inverters HAOS HA 2025.11.2

Log file predbat (3).log

Predbat debug yaml file predbat_debug.yaml

and txt version:

predbat_debug.txt

Nov 17 '25 21:11 gcoan

I was also just about to query how to deal with heat-pump loads in predbat as I have exactly the same situation - load divergence of 147% caused by the heat pump using more than expected and the batteries running out because they weren't charged up enough during the cosy period.

Nov 17 '25 22:11 TCWORLD

@TCWORLD snap! I have put manual charges in tonight for all the Cosy periods, but this is a fudge. I have a feeling I had input_number.predbat_metric_inday_adjust_damping set to 1.1 or 1.2 in the past so that predbat would assume if load had increased today it would continue increasing, but I need to look into this again tomorrow

Nov 17 '25 23:11 gcoan

I'd strongly recommend attempting to get predheat setup inside your predbat system. It essentially resolves this problem by making fairly accurate predictions of heat pump load and mixing that in to the predbat forecast.

Generating some of the required heat loss numbers can be difficult depending on how much data you have coming out of your heat pump system. However, even without entirely accurate historic data you can make a decent guess and use the example "Temperature chart" (https://github.com/springfall2008/batpred/blob/main/templates/example_chart_predheat.yaml_template) to compare the prediction with what "fits" with your system. (Otherwise, you could use the help of your favourite Large Language Model to parse the instructions in the predheat documentation and feed it details of your home for it to generate some decent estimated values).

I've just got predheat hooked-up for this heating season and so far it does seem to be be creating pretty accurate predictions of heating load.

Nov 18 '25 21:11 sammort

@sammort I was going to query about how predheat feeds into predbat as I do actually have it already set up, though I've not really done much playing around with it to optimise things.

The predheat predictions are actually coming out about double the actual usage, yet the predbat plan is under predicting usage, so I'm wondering whether I've configured something wrong - I can't tell whether its actually feeding into the plan or not.

Nov 18 '25 22:11 TCWORLD

So predheat can be making predictions of load that are not being added to predbat's forecasting model. To join-up predheat's forecast with predbat you need to configure predbat as per the load-forecast documentation, specifically you need to add something like this to your predbat apps.yaml

  load_forecast:
    - predheat.heat_energy$external

What's not fully documented (as part of predheat), but what I think is correct, is I've also added a sensor that tracks my ASHP energy use to car_charging_energy, so that bit of config looks like this for me:

  car_charging_energy:
    - 'sensor.myenergi_zappi_xxxxxxx_charge_added_session'
    - 'sensor.ashp_electric_energy'

This excludes historic ASHP load from the predbat load prediction model, so it is not double counting ASHP energy by adding both the historic and forecasted energy. As I say, this isn't explicitly mentioned in the documentation but instinctively feels like the correct configuration and my load forecasts have looked reasonable.

Nov 18 '25 23:11 sammort

What's not fully documented (as part of predheat), but what I think is correct, is I've also added a sensor that tracks my ASHP energy use to car_charging_energy, so that bit of config looks like this for me:

yes that's correct, if your normal house load sensor includes your heat pump energy then that'd need to be subtracted to stop double counting when predheat calculates the heating load.

I've added it to the predheat documentation. Personally I have never tried predheat, it just seemed like too much complexity to setup, and too easy to make wrong assumptions that would result in garbage in/garbage out.

And for me its a moot point anyway forecasting the heat pump energy too accurately, the heat pump will consume on a cold day more than my battery capacity (the batteries ran out at 21:30 today giving half an hour of grid import before the next Cosy period). Its just at the moment with historical load being lower predbat is not predicting that I need so much energy and thus not charging the batteries. Its compounded by inday adjustment not rolling over midnight so I have to put manual force charges in. Once the days_previous history fills up with higher heat pump activity I am sure the predbat forecast will be better

Nov 18 '25 23:11 gcoan

Ok thanks, that explains why it wasn't using the heat pump predictions! I've set both those up (load_forecast and car_charging_energy) and the plan is now reflecting the predicted energy.

~~Now I just need to get predheat to predict accurately. Having some weird issue with target temperature, but I'll move that to a separate issue.~~ Ignore that, I added a new sensor to track heating energy yesterday, so it was detecting 0 days of historic data and thus not loading the target temperature history. Now that's rolled over to a full day of history it seems to be behaving reasonably.

Nov 19 '25 00:11 TCWORLD

For what it's worth predheat seems to be doing a pretty good job now it's set up properly - it's within about 10% over estimating the actual energy usage. With it feeding into the plan, predbat is now correctly charging the battery to full by the end of the mid-day cosy dip.

Nov 20 '25 22:11 TCWORLD

Be interested to see your predheat config. How much did you do the modelling proposed in the setup instructions? It all seemed too much like hard work for me.

At the moment I have just resurrected my Cosy automation and added force charges for all the Cosy periods although I'd prefer to not do this and let predbat work it out for itself.

I have an additional problem that requires this automation; I have two inverters each with batteries but of different sizes (9.5 and 5.2). Each inverter charge rate is 2.4kWh so a 3 hour Cosy period isn't enough to fully recharge the battery either overnight or in the afternoon. Predbat (at present) only has a single global inverter reserve value so I have to put predbat into read only mode, set the reserve on the 9.5 inverter to 27% to keep it quarter full (for the evening peak), then turn read only off when its time to charge the battery. Again, would like Predbat to do more of this for me, but it's another enhancement request.

Nov 20 '25 22:11 gcoan

Ah, I've only got a single inverter/battery setup, and the inverter can do 6kW charge rate so it can top it off in the cosy window.

My predheat config is attached below. For the setup I followed the documentation.

Most of the settings are somewhat obvious - just feed in the appropriate sensors. There are a couple that required calculation, and additionally a couple of data entry ones from the heat pump user manual (I have a 4kW Ideal Logic Air unit hooked up to home assistant using BSB-LAN).

The weather compensation values I got from a chart in the heat pump user manual, and the efficiency vs temperature values came from a datasheet from the manufacturer.

For the heat_loss_degrees setting the calculation given was easy enough to follow (though MathML or picture of the equation added to the docs wouldn't go a miss). Just looked through the HA history and compared the internal (living room) temperature and the outside temperature (heatpump sensor reports this).

$$\begin{eqnarray} T_{in0} &:& \text{Inside temperature start} \ T_{in1} &:& \text{Inside temperature end} \ T_{ext0} &:& \text{External temperature start} \ T_{ext1} &:& \text{External temperature end} \ \Delta h &:& \text{Time in hours} \ \ Loss &=& \dfrac{T_{in1}- T_{in0}}{\Delta h \times \dfrac{(T_{in1} + T_{in0})- (T_{ext1} + T_{ext0})}{2}} \end{eqnarray}$$

e.g for one period, in 5hrs temp dropped from 20.7°C to 18.3°C with a continuous outside temp of 6.4°C, so loss is 0.0366.

The only bit I found problematic was calculation of the heat_loss_watts setting which I couldn't make heads or tails of the description (I started a discussion about it). In the end I just used the estimate in the heat loss survey from the HP installer - that was 3.9kW at 20°C internal and -3°C external, which came to around 171W per degree, so I just used that.

  predheat:
    # Days forward
    forecast_days: 2

    # Days previous is the number of days back to find historical load data
    # Recommended is 7 to capture day of the week but 1 can also be used
    # if you have more history you could use 7 and 14 (in a list) but the standard data in HA only lasts 10 days
    days_previous:
      - 7

    # Gas or heatpump mode ('gas' or 'pump')
    mode: pump

    # External temperature sensor
    external_temperature: sensor.outside_temperature

    # Internal temperature sensor(s)
    internal_temperature: sensor.living_room_temperature

    # Weather data
    weather: weather.met_office_chapel_allerton

    # Sensor with history that monitors the thermostat setting in the house
    target_temperature: sensor.living_room_target

    # Thermostat Hysteresis turn on amount in degrees (default 0.5)
    hysteresis: 0.7

    # Thermostat Hysteresis turn off amount in degrees (default 0.1)
    hysteresis_off: 0.1

    # Heat pump target temperature sensor comes from thermostat and accounts for early turn on.
    smart_thermostat: false

    # Past energy consumption of heating in kWh
    heating_energy: sensor.todays_heating_energy

    # Heating is turned on history
    heating_active: binary_sensor.heating_circuit_active

    # Static heat sources in the house (e.g. people/equipment)
    heat_gain_static: 200

    # House heat loss in degrees per hour per degree temp difference
    #  In 5hrs temp dropped from 20.7 to 18.3 with a continuous outside temp of 6.4
    #   (20.7 - 18.3) / 5 / ((20.7 - 6.4 + 18.3 - 6.4) / 2)
    heat_loss_degrees: 0.0366

    # House heat loss in watts per degrees temp difference
    heat_loss_watts: 171

    # Heating max output (of the radiators), in Watts at delta 50 (BTU / 3.41)
    heat_output: 13745

    # Add up radiator volume + any pipework or expansion vessel
    heat_volume: 133.43

    # Heating max power in Watts
    heat_max_power: 7000
    heat_min_power: 2000

    # Heating cop is the maximum efficiency and will be scaled down based on temperatures
    # put 1.0 for condensing gas boilers, or around 4.0 for heat pumps
    heat_cop: 5.68

    # Current flow temperature setting
    flow_temp: sensor.heatpump_flow_setpoint
    flow_difference_target: 4.5
    
    # Current volume temperature. Ideally this would be sensor in radiator, but trying out return temp.
    volume_temp: sensor.heatpump_return_temp

    # Sets the efficiency of a heat pump in COP at different input temperatures
    heat_pump_efficiency:
      -7: 2.51
      -6: 2.59
      -5: 2.67
      -4: 2.75
      -3: 2.82
      -2: 2.88
      -1: 2.94
      0: 3.0
      1: 3.05
      2: 3.07
      3: 3.09
      4: 3.13
      5: 3.27
      6: 3.51
      7: 3.93
      10: 4.05
      12: 5.14
      14: 5.59
      16: 5.68
      18: 5.68
      20: 5.68

    # Set the weather compensation curve
    weather_compensation:
      -20: 45.0
      -16: 45.0
      -3: 45.0
      10: 34.0
      15: 28.0
      17: 25.0
      20: 20.0

Nov 21 '25 00:11 TCWORLD

Should we make in day adjustement carry forward into the next day until it gets new data to show otherwise?

Nov 22 '25 15:11 springfall2008

Implementation Complete

I've implemented the in-day adjustment blending feature to address this issue. The changes ensure that after midnight, Predbat uses yesterday's in-day adjustment factor until sufficient data is available today, preventing battery depletion in early morning hours.

Changes Made

1. Modified apps/predbat/output.py - load_today_comparison() method:

Fetch yesterday's adjustment (lines 2418-2441): Retrieves yesterday's final in-day adjustment factor from Home Assistant history using get_history_wrapper() and minute_data(). Falls back to 1.0 if no history exists. Also applies min/max capping (0.5-2.0) to yesterday's value for safety.
Calculate blending weight (lines 2447-2455): Implements 3-phase blending:
- 0-3 hours (0-180 minutes): Uses 100% of yesterday's adjustment factor
- 3-8 hours (180-480 minutes): Linear interpolation from 100% yesterday to 100% today
- 8+ hours (480+ minutes): Uses 100% of today's adjustment factor
Apply blended adjustment (lines 2457-2469):
- Computes today's damped factor: (difference - 1.0) * damping + 1.0
- Blends with yesterday's factor: (yesterday * weight) + (today * (1.0 - weight))
- Applies min/max caps (0.5-2.0) to the final result
- Respects the existing calculate_inday_adjustment flag
Enhanced logging (lines 2471-2491): Log messages now show:
- yesterday X% - Yesterday's final adjustment factor
- today Y% - Today's calculated adjustment factor
- blend Z% - Percentage of yesterday being used (100% at midnight, 0% at 8 hours)
New sensor attributes (lines 2508-2520): Added to predbat.load_inday_adjustment:
- yesterday_adjustment: Yesterday's final adjustment as a percentage
- yesterday_weight: Current blend weight as a percentage

2. Updated docs/output-data.md:

Documented the blending behavior in the "In-day load adjustment data" section, explaining the 3-phase transition and new sensor attributes.

How It Works

The blending algorithm uses a linear interpolation during the 180-480 minute window:

yesterday_weight = (480 - minutes_now) / 300.0  # where 300 = 480 - 180

This creates a smooth transition:

At midnight (0 min): 100% yesterday
At 3 hours (180 min): 100% yesterday
At 5.5 hours (330 min): 50% yesterday, 50% today
At 8 hours (480 min): 100% today
After 8 hours: 100% today

Example Log Output

Today's load divergence 130.4 % in-day adjustment 128.88 % damping 0.95x yesterday 125.0 % today 130.4 % blend 60.0 %

This shows:

Raw divergence: 130.4%
Final blended adjustment: 128.88%
Damping factor: 0.95x
Yesterday's factor: 125.0%
Today's calculated factor: 130.4%
Blend weight: 60% yesterday, 40% today

Key Features

✅ Backward compatible: Calculation always runs; application controlled by existing calculate_inday_adjustment flag
✅ State persistence: Uses Home Assistant history, survives restarts
✅ Performance optimized: Only fetches 1 day of history (cached)
✅ Safety caps: Both yesterday's and final blended values capped at 0.5-2.0x
✅ Full visibility: Logging and sensor attributes show all blend components
✅ Smooth transition: Linear interpolation prevents sudden jumps

For Heat Pump Users

This specifically addresses the issue where heat pumps consume more energy on cold days. Previously, after midnight the adjustment would reset to 1.0 (no adjustment), causing the battery to run out. Now:

First 3 hours: Uses yesterday's high adjustment factor (e.g., 130%) immediately
Hours 3-8: Gradually transitions to today's developing adjustment
After 8 hours: Fully using today's data with sufficient samples

This ensures your batteries are charged appropriately during overnight periods when heating demand is high.

Testing Recommendations

Monitor the new sensor attributes:

Check predbat.load_inday_adjustment attributes in Developer Tools → States
Watch the log messages during the first 8 hours after midnight
Verify battery doesn't deplete during early morning hours on cold days
Confirm smooth transition between yesterday's and today's factors

The implementation is complete, tested for syntax errors, and ready for use. It should resolve the battery depletion issues described in this issue.

Nov 22 '25 15:11 springfall2008

I'm using Predheat and the forecast is pretty bang on... today for example forecasted up to now 5.74kWh, measured 5.5kWh.

Nov 23 '25 17:11 OneTwo3D

Thanks Trefor, I need to upgrade to a newer version of Predbat and will try this out, looks to be a great solution to the issue I raised, especially for heat pump prediction.

Got a bunch of stuff to PR in as well!

And thanks to @TCWORLD for the predheat config, this gives me a bump start, I still have my concerns about the complexity of setting this up, but I'll give it a go

Nov 23 '25 21:11 gcoan