Direct Octopus prediction way off compared to Octopus Integration

[pacemaker82]

Describe the bug After setting up the Direct Octopus integration, the best metric sensor prediction is significantly different (much less). You can see in the screenshot below where I enabled, then disabled the direct Octopus connection, there is over £1.00 difference in the prediction.

To install Octopus Direct, I added the 3 new params to apps.yaml (documented here) - I did not comment out anything else - should I? The docs are not clear on that.

Expected behaviour Prediction would be the same between Octopus integration and direct octopus through predbat.

Predbat version

8.16.1

Environment details

• Giv 9.5 kWh AC3
• HA GREEN

Screenshots above

Log file Let me know if necessary.

Predbat debug yaml file Let me know if necessary

[springfall2008]

Are you sure its not because standing charge is accounted for when this is enabled?

[pacemaker82]

Mine is only £0.53 so the math isn’t working out there

[springfall2008]

Mine is only £0.53 so the math isn’t working out there

Depends on if you have one or two standing charges in the plan?

[pacemaker82]

Plan is showing just over the single standing charge (plus some import) (+63p) at midnight

[springfall2008]

Can you capture the plan with and without and save it here, at least the plan HTML but ideally also the debug yaml?

[pacemaker82]

Here is the BEFORE using the Octopus Integration (AFTER coming next. ... ):

Plan HTML:

plan_HMTL_before.txt

Debug YAML

predbat_debug_19_35_00.yaml.txt

[pacemaker82]

Here is the AFTER - so the Direct Octopus is now enabled:

Best price adjusted from £1.72 to £0.76

Plan HTML

plan_HTML_after.txt

DEBUG YAML

predbat_debug_19_40_00.yaml.txt

[springfall2008]

I can see in your 'before' case there are extra low rate slots from 5:30am until 6:30am which are not present in the after case

    2025-03-08T23:00:00+0000: 7.0
    2025-03-09T06:00:00+0000: 7.0
    2025-03-09T06:30:00+0000: 27.16
    2025-03-09T07:00:00+0000: 7.0
    2025-03-09T07:30:00+0000: 7.0
    2025-03-09T08:00:00+0000: 27.16

After:

    2025-03-09T05:00:00+0000: 7.0
    2025-03-09T05:30:00+0000: 27.16
    2025-03-09T06:30:00+0000: 27.16
    2025-03-09T07:00:00+0000: 7.0
    2025-03-09T07:30:00+0000: 7.0
    2025-03-09T08:00:00+0000: 27.16
    2025-03-09T23:00:00+0000: 27.16
    2025-03-09T23:30:00+0000: 7.0

I'm going to hazard a guess that those slots were planned but not used and hence got cancelled by Octopus, but the octopus integration hasn't noticed. Can you confirm if your car actually charged in those extra slots?

[pacemaker82]

Attached is the confirmed Octopus usage from yesterday (it comes in as the 'previous_accumulative_cost' in the Octopus Integration)

I can see for example that 05.30-06.00 is cheap rate:

start: '2025-03-09T05:30:00+00:00' end: '2025-03-09T06:00:00+00:00' rate: 0.070003 consumption: 2.795 cost: 0.2

yesterday-usage.txt

[pacemaker82]

Here is the car charging with the rates from yesterday graphed over the top. Seems well aligned. Each column is a 30 min slot

[springfall2008]

I'm trying to figure out if its a bug in the Octopus Integration of if its actually correct.

What does completed dispatches show today in your octopus integration (binary_sensor.octopus_energy_XXXXXXXX_intelligent_dispatching) and in the predbat version (binary_sensor.predbat_octopus_XXXXXXX_intelligent_dispatch)

[pacemaker82]

It’s off at the moment as I disabled the built in octopus due to inaccuracy. Do you need me to enable it for a day?

[pacemaker82]

Here's what I am seeing between the two dispatches sensors. note: I only enabled the built in again at midnight.

[pacemaker82]

Unsure if this is significant but the two sensors show very different on/off behaviour. This is for 12th March

[springfall2008]

I'll take a look at that difference also, but I think that's maybe related to the update frequency

[pacemaker82]

I installed latest 8.16.4 and re-enabled the direct octopus - this time the predicted best metrics cost has stayed consistent with the octopus integration way - so maybe something got fixed?

[pacemaker82]

I spoke to soon.

Notice the Total Cost. Using Octopus Direct:

Using the Octopus Integration:

[springfall2008]

There appears to be a difference in past costs, can you snapshot the dispatch data from both sensors ?

[pacemaker82]

Intelligent Dispatching.txt

Octopus Intelligent Dispatches.txt

There is no completed dispatches in the octopus direct dispatch sensor

[pacemaker82]

I left Octopus Direct running all night. Here are the two sensors attributes:

Intelligent Dispatching-20250316.txt

Octopus Intelligent Dispatches-20250316.txt

--

I disabled Octopus direct now that I am clear of any charging - my best cost changed from £0.54 w/ Octopus direct, to £1.93 w/ Octopus integration.

Note: interesting that the integration doesn't have any planned dispatches, yet the direct sensor does for tonight. Those aren't in my plan.

[springfall2008]

Thanks, so the Octopus Integration stores the completed dispatches as it seems octopus API doesn't return them from yesterday. I'll do a fix

[pacemaker82]

Thanks - just wondering how that would resolve the issue where the completed dispatches are stored for "today", and the predicted best cost changes dramatically between Direct Vs Integration ?

[cyberkryten]

Unfortunately, the assumption that the completed dispatches show low rate periods is not correct, as confirmed on the Octopus forums after I'd been posting examples: https://forum.octopus.energy/t/pending-and-completed-octopus-intelligent-dispatches/8510/131

They ONLY record car charging periods.

This means that Completed Dispatches can't be used to accurately calculate costs.

There are two clear cases where completed dispatches does not show the correct rate:

1. If a car wasn't charging during a planned dispatch, the rate would be the low one but no completed dispatch is recorded.

2. A Bump Charge at high rate creates a completed Dispatch that is indistinguishable from a low rate period - there's no indication it was a costly bump charge.

[pacemaker82]

Thanks @cyberkryten

I think at the very least, the Octopus Direct method from Predbat needs to reflect the same calculation of "cost so far today" as the Octopus integration does (even if that is technically inaccurate per @cyberkryten advice), so that its predictive model is accurate. I'm still unclear if the changes just released in 8.16.6 actually resolve that or not.

[cyberkryten]

The Octopus docs aren't entirely clear on the 'dispatches' meaning and it seems we all assumed something that wasn't true.

There's an issue open on the Octopus Integration where I've added the same comments - might be worth waiting to see if there's any ideas come from Dave or others of how to do the pricing:

https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/issues/1189#issuecomment-2732668770