Octopus Agile rates not cached after restart

[AndySmithers]

I have Predbat running in a docker container on a Synology NAS. Home Assistant also runs in a Docker container. I have FoxESS hardware. I am using BottleCapDave's Octopus integration with Octopus Agile

With HA and Predbat running correctly I shut down and re-start the NAS. When HA and Predbat containers re-start, Predbat does it's calculation thing and then updates the various charts and plans. HOWEVER, the Octopus Agile rates for today are all set to 0p. This can, of course, convince Predbat to start grid use (since it's free).

How do I force an update of the current day's rates? I've tried simply waiting the 15 minutes for BottleCapDaves integration to pull the rates again. My Octopus integration is showing all the correct rates. Why can't PredBat see theses rates?

[gcoan]

can you share the parts of your apps.yaml where you define the rates and the logfile where this happens.

Predbat shouldn't need to cache the rates, they should be available in Home Assistant from the Octopus integration and unless something zaps them to zero it should all just work OK.

Can you check there wasn't any errors in the Octopus integration on HA startup that might have caused the rates to get cleared out?

[AndySmithers]

The rates section of apps.yaml is unchanged from the default:

 # Energy rates
  # Please set one of these three, if multiple are set then Octopus is used first, second rates_import/rates_export and latest basic metric

  # Set import and export entity to point to the Octopus Energy plugin import and export sensors
  # automatically matches your meter number assuming you have only one (no need to edit the below)
  # Will be ignored if you don't have the sensor but will error if you do have one and it's incorrect
  # NOTE: To get detailed energy rates you need to go in and manually enable the following events in HA
  #       event.octopus_energy_electricity_xxxxxxxx_previous_day_rates
  #       event.octopus_energy_electricity_xxxxxxxx_current_day_rates
  #       event.octopus_energy_electricity_xxxxxxxx_next_day_rates
  # and if you have export enable:
  #       event.octopus_energy_electricity_xxxxxxxx_export_previous_day_rates
  #       event.octopus_energy_electricity_xxxxxxxx_export_current_day_rates
  #       event.octopus_energy_electricity_xxxxxxxx_export_next_day_rates
  # Predbat will automatically find the event. entities from the link below to the sensors
  metric_octopus_import: 're:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_current_rate)'
  #metric_octopus_export: 're:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_export_current_rate)'

  # Standing charge in pounds, can be set to a sensor or manually entered (e.g. 0.50 is 50p)
  # The default below will pick up the standing charge from the Octopus Plugin
  # The standing charge only impacts the cost graphs and doesn't change the way Predbat plans
  # If you don't want to show the standing charge then just delete this line or set to zero
  metric_standing_charge: 're:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_current_standing_charge)'

  # Or set your actual rates across time for import and export
  # If start/end is missing it's assumed to be a fixed rate
  # Gaps are filled with zero rate
  #rates_import:
  #  -  start: "00:30:00"
  #     end: "04:30:00"
  #     rate: 7.5
  #  -  start: "04:30:00"
  #     end: "00:30:00"
  #     rate: 40.0
  #
  # rates_export:
  #  -  rate: 0.15

  # Can be used instead of the plugin to get import rates directly online
  # Overrides metric_octopus_import and rates_import
  # rates_import_octopus_url : "https://api.octopus.energy/v1/products/AGILE-24-10-01/electricity-tariffs/E-1R-AGILE-24-10-01-D/standard-unit-rates/"

  # Overrides metric_octopus_export and rates_export
  # rates_export_octopus_url: "https://api.octopus.energy/v1/products/FLUX-EXPORT-BB-23-02-14/electricity-tariffs/E-1R-FLUX-EXPORT-BB-23-02-14-A/standard-unit-rates"
  # rates_export_octopus_url: "https://api.octopus.energy/v1/products/AGILE-OUTGOING-BB-23-02-28/electricity-tariffs/E-1R-AGILE-OUTGOING-BB-23-02-28-A/standard-unit-rates/"
  # rates_export_octopus_url: "https://api.octopus.energy/v1/products/OUTGOING-VAR-24-10-26/electricity-tariffs/E-1R-OUTGOING-VAR-24-10-26-D/standard-unit-rates/"

  # Import rates can be overridden with rate_import_override
  # Export rates can be overridden with rate_export_override
  # Use the same format as above, but a date can be included if it just applies for a set day (e.g. Octopus power ups)
  # This will override even the Octopus plugin rates if enabled
  #
  #rates_import_override:
  # -  date: '2023-09-10'
  #    start: '14:00:00'
  #    end: '14:30:00'
  #    rate: 112
  #    load_scaling: 0.8

These are the only log entries I get for Predbat:

2025-03-25 16:25:05.013 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.battery_power exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.030 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.battery_cycle exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.075 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.load_power exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.124 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.load_energy exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.150 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.pv_energy exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.312 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.soc_kw_base10 exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.716 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.load_power_best exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:05.792 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.best_load_energy exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored
2025-03-25 16:25:06.069 WARNING (Recorder) [homeassistant.components.recorder.db_schema] State attributes for predbat.plan_html exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

**This is definitely a timing issue since, after about an hour I checked again and all Agile prices were visible in the chart. The only issue is that, for that hour, Predbat may well (and does!) decide that import is free, so it may as well suck as much as possible from the grid. At peak Agile prices, that hurts!

I've repeated the exact same sequence today - stop Predbat and HA in their respective Docker containers and re-start. Exactly the same thing happens.**

[gcoan]

The HA warning messages about Predbat entity sizes can be supressed, details are in the FAQ's.

I'd still look at what is happening to the Octopus sensors and whether there are any errors occurring with those when HA loads, because that's what Predbat uses. It doesn't cache Octopus rates because HA should be providing them to Predbat.

I think the question is why does predbat get zero values from the Octopus sensors when the HA an Predbat containers are stopped and restarted?

Which in turn suggests looking at the Octopus sensors to confirm what happens to them in HA?

Its very strange, I could maybe understand if Predbat started up quicker than HA and thus got zero values if HA isn't initialised yet, but would have thought that far more things wouldn't have worked if HA wasn't up and Predbat would just have errored and picked up the right rates when HA was available in the next Predbat run. Have to admit never seen similar behaviour myself but I'm using HAOS not docker

[springfall2008]

Can you look in developer tools, find the event for the current day rate and check if it has the rates after your HA restart of if its empty? It maybe Octopus integration takes a while to update this event?

You can use Predbat with Octopus API direct, but its not had as much testing as the Bottle Cap Dave one yet