evcc-io
Add persistent cache for solar forecasts
Most solar forecast providers have rather strict API rate limits, doing a few restart cycles when testing out configuration changes can easily cause evcc to hit the rate limit, having to go without solar forecasts for the next 12-24 hours.
This PR:
- Fixes a bug in the settings deletion logic
- Adds a persistent solar forecast cache stored in the settings table
Expands on functionality in
- https://github.com/evcc-io/evcc/pull/21971
Cache keys use the following components to ensure consistent & unique naming:
- forecast provider name
- forecast configuration checksum
On read the logic ensures that the data has been written by the same evcc version and the cached data is not older than the fetch interval.
Stale forecast cache settings keys will be expired once on startup.
Example log output:
[..]
Jun 22 09:01:56 abundantia evcc[3449784]: [solar-cache] DEBUG 2025/06/22 09:01:56 cache updated: stored 49 rates
[..]
Jun 22 12:01:56 abundantia evcc[3449784]: [solar-cache] DEBUG 2025/06/22 12:01:56 cache updated: stored 49 rates
[..]
Jun 22 14:54:39 abundantia systemd[1]: Stopping evcc.service - evcc...
Jun 22 14:54:39 abundantia systemd[1]: evcc.service: Deactivated successfully.
Jun 22 14:54:39 abundantia systemd[1]: Stopped evcc.service - evcc.
Jun 22 14:54:39 abundantia systemd[1]: evcc.service: Consumed 1min 948ms CPU time.
Jun 22 14:54:41 abundantia systemd[1]: Started evcc.service - evcc.
Jun 22 14:54:41 abundantia evcc[3458503]: [main ] INFO 2025/06/22 14:54:41 evcc 576055b7 (576055b7)
[..]
Jun 22 14:54:45 abundantia evcc[3458503]: [solar-cache] DEBUG 2025/06/22 14:54:45 cache valid: age 2h52m48.463051405s, 49 rates
Jun 22 14:54:45 abundantia evcc[3458503]: [solar-cache] DEBUG 2025/06/22 14:54:45 cache hit: returning 49 rates
Jun 22 14:54:45 abundantia evcc[3458503]: [solcast] DEBUG 2025/06/22 14:54:45 loaded 49 rates from cache
Fixed the delete error plus another one of same pattern in https://github.com/evcc-io/evcc/commit/4406b2399963d84a619fda01b4cd723ccdf1ef2c
The issue with this PR is that we're skipping any tariff creation/ validation step that might happen in run. For example, if we're depending on a valid user name, that error would only occur some 48 hours later. As much as I like the caching idea, I'm not sure that's a good idea.
For example, if we're depending on a valid user name, that error would only occur some 48 hours later.
Can you share what a problematic pattern would be?
If you change any configuration settings, e.g. the token for solcast
https://github.com/evcc-io/evcc/blob/6de801f8613847c183c78b761e7935cd3b9d8f95/templates/definition/tariff/solcast.yaml#L39-L43
this would change the configuration hash and cause a cache miss on startup, showing you any potential validation errors the change has caused.
The only way I see this could mask issues is if you change the token (or related settings) in the forecast providers account management and then restart evcc, which would only start showing the problem at the first interval refresh.
Did I miss any other path?
What do you think about making the feature opt-in via a configuration setting in the solar tariffs? Or alternatively provide a command to display/wipe the cache and call that out in the logs when the cache is first loaded?
Either change should reduce chances of users being surprised.
@terrorobe can we please clarify a few points:
- When starting, we're evaluating all config, e.g. username/password. This PR removes that step if cache exists? I think that's fair to do, if we have cached data we can assume that there was a working config. Point accepted.
- What's the contract for not updating tariff/cache on start? I.e. under which condition should we expect the tariff to execute the login/ data retrieval?
- Last point: I've noticed you've basically touched every tariff to add caching. Wdyt: would it be feasible to add a "proxy" cache that just wraps the tariff? When it thinks fit (see question before) it could return cache data, if not it could eventually initialize the tariff and return tariff data. Seems this might be a less invasive solution providing clearer separation of concerns.
Does that make sense?
I explored the proxy pattern.
There's two main points where we need to decide on how to proceed.
Starting to tick
When cached data is available, we want to delay the first fetch of the API to the remaining interval of the cached data, i.e. interval 3h, cached data is 2h old, delay first fetch for 1h.
Currently, each tariff will start to tick implicitly from its initialization. To allow for delayed initial fetches we'd either need to delay the creation of the tariff or push the tick start from initialization to the first Rates() call.
Do you have any preferences or alternate approaches to tackle this?
When to cache
Rates() will be called at the site interval, by default 30s. The rates themselves only update every 1-3h for most solar providers.
The most obvious approach for the proxy to update the cache is to do so during the Rates() call.
Currently, the tariff interface doesn't offer any information if/when the rates have changed the last time which means that we either need to detect changes in the Proxy or SolarCacheManager, or alternatively - just write the data to the cache whenever Rates() is called.
Should we expand the tariff interface to provide information about when the data was last updated? Or would you go a different route?
There's two main points where we need to decide on how to proceed.
Good questions. However, they apply to the per-tariff-cache pattern just as well ;)
When cached data is available, we want to delay the first fetch of the API to the remaining interval of the cached data, i.e. interval 3h, cached data is 2h old, delay first fetch for 1h.
Can we assume that our main painpoint is subsequent restarts? Then I see two options:
- start tick (and tick equals actually initialising the tariff at all) once we have 1h of missing data
- better: read the tariffs
intervalsetting and use that instead of the fixed 1h
The most obvious approach for the proxy to update the cache is to do so during the Rates() call.
Agreed. For your question I'd go with yagni- don't over-optimise. Either
- store on every call (maybe not)
- do a quick hash (over-optimise)
- checkpoint storing to doing so every 5m and of course after first update
Wdyt?
Original question I missed:
What's the contract for not updating tariff/cache on start? I.e. under which condition should we expect the tariff to execute the login/ data retrieval?
Currently this will happen on any of:
- too old (> 24 hours: no cache, block on immediate fetch, > 1 hour: serve from cache, trigger background fetch)
- changed config
- changed evcc version
I'm not sure if we need to provide a more concrete interface for forcing a refetch. Deleting the cache key or (noop) modifying the config would be the currently available mechanisms.
read the tariffs interval setting and use that instead of the fixed 1h
We currently don't have a good way to read the default from the provider. Opted for an implicit 1 hour default interval for now unless you think there's value in being more correct in case users provide a much longer intervals.
IMO this should be fine since it'll cover frequent restart cycles during config experiments just fine. And during upgrades users will need to refetch during "inopportune" times anyhow.
do a quick hash (over-optimise)
opted for that to prevent users from being unlucky. If that proves to be too expensive we can either augment or replace it with a time based approach.
I'm also pondering if we should surface the cache as config setting for price tariffs since prices can be as important, but I guess this is dependent on user need. Have there been many stories of unavailable price APIs causing user visible problems after a restart?