Energy management/calculations

[jlaur]

I'm creating this issue to start brainstorming ideas which may or may not grow into something. For starters, I will share some unstructured thoughts, which could grow into something structured and potentially into something that could be designed and implemented.

Let me start out by sharing some links which can shed some light into where I'm coming from with this:

• https://github.com/openhab/openhab-addons/pull/13416#discussion_r980432359
• Dishwasher price calculation automation
• Meter readings from Kamstrup Omnimeter with Pow-K+

My initial need is integration to services providing future energy prices. After that, having these prices, I would like to be able to perform calculations. These calculations should be implemented once with a common interface, no matter from which service the prices were obtained. I wonder, from an architectural point of view, if this already means something is needed within openhab-core, since addons cannot depend on other addons? For example, core could provide some interfaces and calculations, while addons could integrate various API's implementing such interface.

Now really brainstorming/dreaming. For the dishwasher example, I logged energy consumption during our most frequently used program and manually mapped that into a timetable in a rule. Considering being able to select the last run specific program on some device (having an energy consumption channel which can be persisted) and map that into another timeslot to calculate the price of that, or having the ideal timeslot calculated automatically within some boundaries. I guess this is just a use-case for the price integrations and calculations and an application which should be built outside of openHAB itself, but just wanted to mention this, so that the parts needed for this could emerge.

I now also have almost real-time logging of the power consumption in my house (see third link above) provided a current power (W) as well as accumulated energy consumed (kWh). The accumulated value is updated once per hour, and current power is updated every 10 seconds. With this data I would like to be able to create a graph like this (screenshot from AMS reader):

For this some post-processing is needed, since I receive the kWh data like this:

2023-01-07 21:00:07.940 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Omnipower_Accumulated' changed from 14609.85 kWh to 14610.24 kWh

i.e. as totals, not hourly contributions. So (14610.24-14609.85) kWh = 0.39 kWh from this log example, which is shown as last bar on the graph above. I'm not sure what is the best approach for doing this, but again it would be nice with something consistent and reusable.

I will update this issue with additional thoughts and knowledge from all of you.

I'm currently considering providing a small binding fetching data from EnergiDataService which is a Danish service providing prices. Yesterday I was also able to receive the same prices from ENTSO-E, but in EUR. So this could reach a broader audience, but would additionally require integration to online currency exchange rates to have conversion to local currencies. Currency question: Do we have any kind of currency handling in openHAB?

[lsiepel]

As other tools as Grafana are way better in visualizing data, i have seen many users do something like this (including myself):

1. Persist an openHAB channel with energy price information (x euro per kw/h from datestamp x)
2. Persist power usage. This can be done at several levels; per device, or on total (e.g. P1 monitor on smart meter), usually some rule is involved for post-processing as most bindings/smart meter only record running totall.
3. Tie data from 1 and 2 together with plain simple SQL and display in Grafana and you have clear insights in hourly costs or even costs of a wash machine run.

But that is just historical. What would the future calculations need to do? Assist with deciding the provider to use? or automated decission to start or delay a power consuming task? Prognoses based on your historicall usage only is not enough, you would need a lot more parameters to train a model.

/me: subscribed to this topic :-)

[jlaur]

What would the future calculations need to do?

See the dishwasher rule example I've provided. I simply program the dishwasher to be finished 7:00 in the morning. The rule will then detect that this has been scheduled and will then calculate the optimal period considering how much power it uses during the program and the future prices. For example, my most used program will use the most power approximately 1:30 into the program and then for the next hour after that. Therefore, for best results, the program needs to be mapped. Assuming linear power consumption for the duration of the program will give very different results.

Assist with deciding the provider to use? or automated decission to start or delay a power consuming task? Prognoses based on your historicall usage only is not enough, you would need a lot more parameters to train a model.

Or something simpler. For example, for the washing machine we use a few more different programs than for the dishwasher, so in this case I haven't mapped any programs yet. However, I have persisted all parameters including the program types and energy consumption. So in theory I could select the last 60 °C cottons program, and the mapping of the program could be done automatically. I could then move this into the future, and the price could be calculated from a period, or the period could be calculated from the lowest price.

[lsiepel]

Feeding tasks with details (e.g start/end boundaries, task duration, energy usage profile) to this ‘energy management service’ is one thing(demand). The other can be a bit more complex. The prices from external providers like you said can be obtained. But what about your own energy generators? Solar panels? That may need some weather prognoses etc? Home batteries? May need some expected battery drain.

Prices here (Netherlands) are sort of fixed, and you only pay for the annual nett (from and to the grid substracted) result of power usage. Because of this method, scheduling is not yet important, but it will as these rules have changed per 2025.

What constraints would the internal scheduler have?

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/bringing-electricity-information-from-eloverblik-dk-and-energidataservice-dk-into-openhab/143470/4

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/entsoe-e-binding-for-nordpool-spot-prices/143833/4

[masipila]

This might be of interest to the readers of this thread: https://community.openhab.org/t/control-a-water-heater-and-ground-source-heat-pump-based-on-cheap-hours-of-spot-priced-electricity/136566/13

And this: https://community.openhab.org/t/entsoe-e-binding-for-nordpool-spot-prices/143833/5

Cheers, Markus

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/1

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/4

[masipila]

@jlaur , here are the calculations that I'm currently using with this solution: https://community.openhab.org/t/control-a-water-heater-and-ground-source-heat-pump-based-on-cheap-hours-of-spot-priced-electricity/136566/13

I have just recently improved my own solution since we (finally) got our electric vehicle. The content below is therefore is fresh from the oven and more advanced than what I have published in the link above.

1. Fetching the spot price and persisting it as a future-timestamped time series A pre-requisite for the the calculations below is that the day-ahead spot prices have been fetched and persisted as a future-timestamped time series. I fetch the data from Entso-E API but I won't cover it in this comment since the title of this issue is about the energy calculations, not about fetching the spot prices. My full code for fetching the spot-prices is available in the link above.

2. Finding the cheapest consecutive N-hour window within a given time range Use cases: I use this method to find the cheapest possible consecutive window for water boiler and charging the car. My rationale for requiring the period to be consecutive hours is that:

• Water boiler: I do not want to interrupt the water heating process of the boiler. I always allow sufficient amount of hours so that the boiler will heat the water until it hits the max temperature of its own thermostat.
• Charging the electric vehicle: I haven't actually tried, but I don't think the car would like that the charging would bounce on and off. I anyway need to charge the car only 2-4 hours per night so I find the cheapest window and charge it one go.

I originally used this so that I was searching the cheapest hours from midnight to midnight, but since the electric vehicle came to the picture, I'm now searching the cheapest window between 21 and 06 so that the car will always be ready in the morning.

/**
 * Finds the cheapest 'num' length window on a given range and prepares the on/off control points.
 *
 * @param Date start
 *   Start of the time range. Hour starting at 'start' is included in the range.
 * @param Date stop
 *   Stop of the time range. Hour starting at 'stop' is not included in the range.
 * @param int num
 *   Length of the window to find.
 *
 * @return array
 *   Array of point objects.
 */
function allowCheapWindow(start, stop, num) {
    console.log('spot-price.js: Searching cheapest window...');
    let cheapestSum = 0;
    let startIndex = 0;

    // Read the spot prices from the database. Sort by datetime.
    const prices = influx.getPrices(start, stop);
    prices.sort((a, b) => (a.datetime > b.datetime) ? 1 : -1);

    for (let i = 0; i <= prices.length - num; i++) {
        let sum = 0;
        // Calculate the sum of the hourly prices for the current 'num' hours.
        for (let j = i; j < i + num; j++) {
            let point = prices[j];
            sum += point.value;
        }
        // Initial value for the cheapestSum.
        if (i == 0) {
            cheapestSum = sum;
        }
        if (sum < cheapestSum) {
            cheapestSum = sum;
            startIndex = i;
        }
    }
    console.log('spot-price.js: Cheapest window starts at index: ' + startIndex + ', sum: ' + cheapestSum);

    // Prepare control points.
    let points = [];
    for (let i = 0; i < prices.length; i++) {
        let value = 0;
        if ((i >= startIndex) && (i < startIndex + num)) {
            value = 1;
        }
        let point = {
            datetime: prices[i].datetime,
            value: value
        }
        points.push(point);
    }

    // Return calculated control points
    return points;
}

3. Populating the rest of the day with "OFF" control points As you can see from the code above, the concept is to generate control points (either 1 or 0) for each hour of the day. I then have a rule that runs at every full hour to see if the actual Switch Item needs to be toggled on or off.

Anyway, the fact that I search for the cheapest car charging window between 21-06 means that I don't have control points from 07:00 onwards. This method allows me to populate 0 value ("off") control points from 07:00 onwards.

/**
 * Blocks the given range regardless of the price.
 *
 * @param Date start
 *   Start of the time range.
 * @param Date stop
 *   Stop of the time range.
 *
 * @return array
 *   Array of point objects.
 */
function blockHours(start, stop) {
    let points = [];

    // Convert from milliseconds to hours
    diff = (stop - start) / 1000 / 3600;

    for (let i = 0; i < diff; i++) {
        let dt = new Date(start);
        dt.setHours(dt.getHours() + i);
        let point = {
            datetime: dt.toISOString(),
            value: 0
        }
        points.push(point);
    }
    return points;
}

4. Finding the individual cheapest hours from a given range, cheap hours do not need to be consecutive. Use case: Heating of the house

/**
 * Finds 'num' cheapest hours withing the range and prepares the on/off control points.
 *
 * @param Date start
 *   Start of the time range.
 * @param Date stop
 *   Stop of the time range.
 * @param int num
 *   Number of cheap hours to find.
 *
 * @return array
 *   Array of point objects.
 */
function allowCheapHours(start, stop, num) {
    console.log('spot-price.js: Searching cheapest hours...');
    let points = [];

    // Read the spot prices from the database and sort by price.
    const prices = influx.getPrices(start, stop);
    prices.sort((a, b) => (a.value > b.value) ? 1 : -1);

    // Prepare control points.
    for (let i = 0; i < prices.length; i++) {
        let value = (i < num) ? 1 : 0;
        let point = {
            datetime: prices[i].datetime,
            value: value
        }
        points.push(point);
    }

    // Sort points by datetime.
    points.sort((a, b) => (a.datetime > b.datetime) ? 1 : -1);
    return points;
}

5. Finding the individual expensive hours from a given range, hours do not need to be consecutive. Use case: Like above, but for some devices somebody might want to avoid the most expensive hours.

/**
 * Finds 'num' most expensive hours withing the range and prepares the on/off control points.
 *
 * @param Date start
 *   Start of the time range.
 * @param Date stop
 *   Stop of the time range.
 * @param int num
 *   Number of cheap hours to find.
 *
 * @return array
 *   Array of point objects.
 */
function blockExpensiveHours(start, stop, num) {
    console.log('spot-price.js: Searching most expensive hours...');
    let points = [];

    // Read the spot prices from the database and sort by price.
    const prices = influx.getPrices(start, stop);
    prices.sort((a, b) => (a.value > b.value) ? -1 : 1);

    // Prepare control points.
    for (let i = 0; i < prices.length; i++) {
        let value = (i < num) ? 0 : 1;
        let point = {
            datetime: prices[i].datetime,
            value: value
        }
        points.push(point);
    }

    // Sort points by datetime.
    points.sort((a, b) => (a.datetime > b.datetime) ? 1 : -1);
    return points;
}

6. Finding the individual cheapest hours for heating, balancing the heating throughout the day. Use case: heating the house when it's very cold and heating must be allowed also during the daytime even if the day hours would be more expensive than some night hours. In other words: when it's -10 C or colder, our the inside temperature of our house will drop too much during the day if I only heat it during the night. In yet another words: I also need to find some "less expensive" hours from the daytime.

First of all, I calculate the needed amount of heating hours based on the weather forecast. The weather forecast has been persisted to a measurement 'fmi_forecast_temperature' with future timestamps.

(Side note: In my own implementation I actually consider both temperature and wind speed of the weather forecast because the house cools down more when it's windy. I use the so called wind chill factor, the "feels like" temperature you can see in weather forecasts, see https://en.wikipedia.org/wiki/Wind_chill#North_American_and_United_Kingdom_wind_chill_index)

I have searched empirically the house-specific formula that is basically "when tomorrow's average temperature is X, I need to allow Y heating hours for the heat pump".

When the number of needed heating hours (Y above) is known / has been calculated, I can use the method below for finding the hours when I will allow the heat pump compressor to be on. Most of the time during this winter I have only heated the house during the night hours (number of slices has been 1) but on very cold days I have split the day to three 8 hour slices and searched the cheapest hours from each slice.

/**
 * Exports.
 */
module.exports = {
    getForecastTemp: getForecastTemp,
    calculateNumberOfHours: calculateNumberOfHours,
    determineHours: determineHours
};

/**
 * Reads forecasted temperatures from the database and calculates an average.
 *
 * @param Date start
 *   Start of the time range.
 * @param Date stop
 *   Stop of the time range.
 *
 * @return float
 *   Average temperature for the range.
 */
function getForecastTemp(start, stop) {
    console.log('nibe.js: Calculating forecasted average temperature...');
    influx = require('kolapuuntie/influx.js');
    const csv = influx.getPoints('fmi_forecast_temperature', start, stop);
    const points = influx.parseCSV(csv, 5, 6);
    let sum = null;
    let avg = null;
    for (let i = 0; i < points.length; i++) {
        sum += points[i].value;
    }
    if (points.length) {
        avg = sum / points.length;
    }
    console.log('nibe.js: average temperature: ' + avg);
    return avg;
}
/**
 * Calculates number of needed hours for given average temperature.
 *
 * @param float temperature
 *   Average temperateure for the day.
 *
 * @return float
 *   Number of hours the heat pump should be allowed to run.
 */
function calculateNumberOfHours(temperature) {
    console.log('nibe.js: Calculating number of ON hours for Nibe...');

    // Early exit if temperature is null.
    if (temperature == null) {
        console.warn('nibe.js: No temperature given! Number of needed hours defaulted to 24!');
        return 24;
    }

    // Calculate curve based on two constant points.
    // y = kx + b
    // x = temperature, y = number of needed hours.
    const p1 = {
        x : -20,
        y : 22
    };
    const p2 = {
        x: 20,
        y: 2
    }
    const k = (p1.y-p2.y) / (p1.x-p2.x);
    const b = p2.y - (k * p2.x);
    console.debug('nibe.js: y = ' + k + 'x + ' + b);

    let y = k * temperature + b;
    if (temperature < p1.x) {
        y = p1.y;
    }
    if (temperature > p2.x) {
        y = p2.y;
    }
    console.log('nibe.js: Number of needed hours: ' + y);
    return y;
}
/**
 * Calculates the on/off hours for the heatpump.
 *
 * @param Date start
 *   Start of the time range.
 * @param Date stop
 *   Stop of the time range.
 * @param int num
 *   Number of hours the heat pump must be on.
 * @param int slices
 *   Divide day into this many slices to balance heating during the day.
 *   Caller is responsible for ensuring that slices is > 0 and < length of the time range.
 *   Example: 2 would result in 2 x 12h slices if the range is 24 hours
 * @paramn float min
 *   Minimum share of heating hours each slice must have.
 *   Caller is responsible for ensuring that the value is a sensible float between 0 and 1.
 *   Example:
 *     Let's say 10h of heating is required and the day is split into 2 slices.
 *     Value 0.1 (10%) means that both slices must have at least 0.1x10h = 1 hour of heating.
 *     The minimum number of hours per slice is rounded down.
 *
 * @return array
 *   Array of point objects.
 */
function determineHours(start, stop, num, slices, min) {
    console.log('nibe.js: Determining on/off hours for Nibe...');
    // console.log("start: " + start);
    // console.log("stop: " + stop);
    // console.log("num: " + num);
    // console.log("slices: " + slices);
    // console.log("min: " + min);
    let selectedHours = [];

    // Read the spot prices from the database and slice the day.
    const prices = influx.getPrices(start, stop);
    const priceSlices = slicePrices(prices, slices);

    // Pick min number of hours from each slice to the final array, but at least 1 per slice.
    let minPerSlice = Math.floor(num*min);
    if (minPerSlice == 0) {
        minPerSlice = 1;
    }

    console.debug('nibe.js: Minimum number of hours for each slice: ' + minPerSlice);
    for (let i = 0; i < priceSlices.length; i++) {
        for (let j = 0; j < minPerSlice; j++) {
            selectedHours.push(priceSlices[i][j]);
            priceSlices[i].splice(j, 1); // Removes price from the slice since it's used.
            num--;
        }
    }

    // console.log("selected hours: ");
    // console.log(selectedHours);

    // Rest of the needed hours can be chosen freely based on the price.
    const merged = mergeSlices(priceSlices);
    // console.log("merged: ");
    // console.log(merged);
    const rest = merged.slice(0, num);
    // console.log("rest: ");
    // console.log(rest);
    selectedHours = selectedHours.concat(rest);
    // console.log("selected");
    // console.log(selectedHours);
    const unselectedHours = merged.slice(num);
    // console.log("unselected");
    // console.log(unselectedHours);
    // Prepare control points.
    const points = preparePoints(selectedHours, unselectedHours);
    // console.log("points");
    // console.log(points);

    return points;
}
/**
 * Slices the spot prices array.
 *
 * @param array prices
 *   Array of spot price objects.
 * @param int slices
 *   Number of slices the 'prices' array should be split.
 *
 * @return array
 *   Array of sliced spot price arrays. Each slice is sorted by spot price.
 */
function slicePrices(prices, slices) {
    const priceSlices = [];
    const n = prices.length;
    const start = 0;
    const length = Math.ceil(n/slices);
    for (let i = 0; i < slices; i++) {
        let slice = prices.slice(i * length, (i+1) * length);
        // Sort slice by prices.
        slice.sort((a, b) => (a.value > b.value) ? 1 : -1);
        priceSlices.push(slice);
    }
    return priceSlices;
}

/**
 * Merges slices to one array.
 *
 * @param array priceSlices
 *
 * @return array
 *   Merged array.
 */
function mergeSlices(priceSlices) {
    let merged = [];
    for (let i = 0; i < priceSlices.length; i++) {
        for (let j = 0; j < priceSlices[i].length; j++) {
            merged.push(priceSlices[i][j]);
        }
    }
    // Sort by prices.
    merged.sort((a, b) => (a.value > b.value) ? 1 : -1);
    return merged;
}
/**
 * Prepares the control points to be written to the database.
 *
 * @param array selectedHours
 * @param array unselectedHours
 *
 * @return array
 *   Array of point objects.
 */
function preparePoints(selectedHours, unselectedHours) {
    let points = [];
    // Value 1 for the selected hours of the day.
    for (let i = 0; i < selectedHours.length; i++) {
        let point = {
            datetime: selectedHours[i].datetime,
            value: 1
        }
        points.push(point);
    }
    // Value 0 for the unselected hours of the day.
    for (let i = 0; i < unselectedHours.length; i++) {
        let point = {
            datetime: unselectedHours[i].datetime,
            value: 0
        }
        points.push(point);
    }
    // Sort by datetime
    points.sort((a, b) => (a.datetime > b.datetime) ? 1 : -1);
    return points;
}

Cheers, Markus

p.s. I have previously published the code with the following license in the community post linked in the beginning of this comment. It's still my copyright so if parts of this code would end up in openhab, it can be licensed with the license that is used with openhab.

/**
 * Copyright (c) 2022 Markus Sipilä.
 *
 * Permission to use, copy, modify, and/or distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
 * REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
 * AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
 * INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
 * LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
 * OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 * PERFORMANCE OF THIS SOFTWARE.
 */

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/tibber-binding-get-best-price-items-like-awattar-binding/144223/9

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/tibber-binding-get-best-price-items-like-awattar-binding/144223/17

[mstormi]

See my proposal at https://community.openhab.org/t/tibber-binding-get-best-price-items-like-awattar-binding/144223/18

[kaikreuzer]

Hey guys, I'm now jumping on this train as well, so count me in! You've all already done amazing work (I'm especially in awe by @masipila rule logics). I clearly see the energy topic to be a hot one and we should indeed think of new additional concepts to better support such use cases. FWIW: I have 2 PV systems, planning a 3rd one with a small battery, 2 wallboxes with EVs, a warm water heatpump, electric heating and just subscribed to Tibber (contract starts July 1st). So you see that I have some "demand" for clever energy management. 😆

I tried to read most of your ideas/discussion here, but as it is quite a lot, I certainly will have missed some aspects.

Let me nonetheless try to summarize what I understood what we would ideally need:

1. A mechanism to store future energy prices (and e.g. also weather forecast data)
2. A way to declare load curves for dedicated devices
3. A way to declare and/or calculate future demand of a device (i.e. heating must run 5 hours within the next 12 hours, EV must be charged to 80% by noon, dishwasher must be finished by 7am, etc.)
4. A central controller that mixes this all together and activates/controls the different devices

Adding PV systems to the solution increases the complexity by another level, since we would additionally need

5. Live information about current power consumption
6. Live information about PV power
7. Improved controller logic that considers tariff prices in combination with (cheap) PV excess power - with the difficulty, that excess power is limited, so load selection must prioritize and weigh the different loads against each other
8. Support of batteries as a "negative" load to apply when grid costs are high.

Did I miss anything crucial? Sounds already like quite some endeavor... 😲

[lsiepel]

Don't want to oversimpify, but for on a high level, i think such a system should consistent of three modules;

1. Demand; device x (like a EV) demands to use a powerprofile y (like 5kw for 5 hours) with z constrains (like finishe before 8:00)
2. Provider; energy providers deliver energy according to a costprofile (like between 8:00 and 10:00 for 1 euro), your own PV system can be a energyprovider with a energy profile (like between 10 - 15:00 so much kwh for 0 euro) or a home battery could also be modeled as a energyprovider.
3. Engine; taking all the demands and providers into calculation according to some user set limits and preferences. (like max concurrent kw/h, timing, look for cheapest or for max self provided or whatever user defined preferences)

Edit: 4. Dashboard/Insights: The system should run fully automated, but i think it should also have an area where it can supply charts / insights. As with most energy related systems this subject deserves its own area/module.

[kaikreuzer]

This simple demand modelling might work for EVs and heatpumps, but not so well for dishwashers, washing machines and tumble dryers - to schedule them well, you must probably consider the load more fine-grained than as a fixed power value (that's at least what I understand from @jlaur's solution). For dealing with excess power, I also think that the demand requires some "priority levels" as @mstormi mentioned here with the SG ready modes. Wrt dashboards: Yes, I agree - it would be nice to have suitable widgets and whole pages for the Main UI that directly support a good energy overview out of the box.

[jlaur]

Let me nonetheless try to summarize what I understood what we would ideally need:

• A mechanism to store future energy prices (and e.g. also weather forecast data)

Yes, this would come in very handy for these use-cases. I guess we could reboot openhab/openhab-core#844 and the implementation proposal openhab/openhab-core#3000.

This simple demand modelling might work for EVs and heatpumps, but not so well for dishwashers, washing machines and tumble dryers - to schedule them well, you must probably consider the load more fine-grained than as a fixed power value (that's at least what I understand from @jlaur's solution).

Correct, a poor man's solution for not having any PV systems, EV's or heat pumps to play with is to optimize scheduling of the dishwasher during the night. 😄

I also just added another small calculation example for some tumble dryer price feedback: https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/5 - taking advantage of calculations currently implemented in openhab/openhab-addons#14376 - specifically https://github.com/openhab/openhab-addons/blob/e6fbfcf04aaf2fd130601534218a8e1d15825225/bundles/org.openhab.binding.energidataservice/src/main/java/org/openhab/binding/energidataservice/internal/PriceCalculator.java

My initial thought was to have such calculations consolidated in some central place so they could be reused for all bindings providing electricity price information. This could be done by providing an interface that such bindings could implement to deliver the needed data.

You are right this is more fine-grained, but perhaps the calculations would actually be quite similar, if not the same, for EV/heat pump use-cases. One difference might be only having a need for running "full hours", but this seems like a simplification, i.e. rounding. At least I would appreciate to have the fined-grained use-cases included in the design we come up with.

This is of course only a small part in the big puzzle.

[mstormi]

It's complex because there's different types of load. Many you can determine (like a dish washing will always take 1.5kWh), some you cannot in neither amount (like a car that constantly charges at say 11 kW but how to do the math if you don't have access to the SoC?) or power requirements (like an inverter heat pump that's consumption will vary a lot). I sort of solved that in my EM system, but due to those different natures it's not 100% consistent/congruent and still somewhat a compromise. Working with hour chunks and priority levels corresponding to SGready modes turned out to be an approach that seems to be working well for the time being because it's the 'natural' interface into heat pumps, and can be applied to EV charging as well. Essentially I set the base level depending on power tariff of the hour but I 'escalate' to a higher level when there's excess PV power. I made the number of hours for levels 1,3 and 4 (2 is "normal") user-configurable, note that's key to applicability. All, please take another look at my post @kai linked to and the details on the model I laid out.

BTW I'm happy to share what I implemented when it's for a good purpose like here. Feel free to download the demo of my EMS and extract the scheduling code to see what I did, it's implemented in rules DSL.

On live data from inverters, it's needed yes but don't put it in the mix here that might kill this issue's scope. Just assume there's (group) items that'll provide values for generation, feedin and consumption. Same on dashboard. Yes please let's join forces. It's easy to spend helluvalot of time on UI design. There's a nice widget https://community.openhab.org/t/animated-energy-widget/133510/60 worth starting with. But it's out of scope for this issue alone here, too.

[jlaur]

To be considered just as a side note: Regarding prices I'll just add that openhab/openhab-addons#14529 has been introduced to avoid redundant VAT calculations. This is probably one of the first "finance" contributions. The next thing that comes to mind could be modelling currencies - currently Number is used for prices without any currency since this is outside the scope of UoM. With the right concept, we could implement currency exchange services. This would be interesting for an ENTSO-E binding since it provides prices only in EUR.

[lsiepel]

s power, I also think that the demand requires some "priority levels" as @mstormi mentioned here with the SG ready modes.

I only tried to model the basics, offcourse the powerprofile can be very detailed. Maybe even a minute by minute power usage table for the device. This model won't prevent those details.

In my mind, the model has enough room for something like a home battery or heat pump to get rid of the excess power. Just schedule a 'device' with constraint that it is not allowed to cost more then 0 money and uses x kw/h. But i also see room for priority as a constraint. There should be even more constraints, i didn't meant it as a limited list, just as a model. The exact details / options / preferences need to be determined, i think it might also be something that iteratevliy may be extended.

The main question is how such feature would fit in OH. I can see the main scheduling engine that takes all the user preferences / calculations etc into account to be somewhere in core. Maybe the device-that-needs-to-be-scheduled can be something like a Thing in from binding. With some tags you might add some metadata to the Thing so it can be picked up by the scheduling engine. But how would you then model the powerprofile? Provide some json data by a channel? Maybe a bit creative ;-) The same for energyprovides. You might use a Thing from a binding, add some tags to it and/or require some channels to be present to get it recognized as a 'offical energyprovided' that can be picked up by the core scheduling engine.?

Or do you guys think of some whole new concept?

[kaikreuzer]

I guess we could reboot https://github.com/openhab/openhab-core/issues/844 and the implementation proposal https://github.com/openhab/openhab-core/pull/3000.

Yes, I was also planning to look a bit closer into that and what it would mean for future prices.

we could implement currency exchange services

Interesting idea, but I would leave it out of scope for the start - this sounds like another complex feature on its own.

BTW I'm happy to share what I implemented when it's for a good purpose like here. Feel free to download the demo of my EMS and extract the scheduling code to see what I did, it's implemented in rules DSL.

Thanks for the offer, I'll try to have a look. For our controller, I guess we might end up implementing it in Java directly then, but Rule DSLs are then pretty close already.

I only tried to model the basics

Yes, I agree that we should start slowly and cover the basics first - in a way that can be extended later on for more complex cases.

The main question is how such feature would fit in OH. Or do you guys think of some whole new concept?

Difficult to say. I don't think we will be able to squeeze everything into the existing concepts, so yes, some new kind of construct might be necessary. I also thought about having it as a feature of a Thing, but that feels as if it is going against the architecture of openHAB as it would prevent providing power profiles for "logical devices", i.e. things that people might only model as items (and have it hooked up to something through HTTP or whatever). I could imagine that we could have something like a "power description", similar to a "state description" for items. We could then have a PowerDescriptionProvider interface and various implementations for this, all delivering the same kind of information that would be used by the PowerController that operates on the items that have such a "power description". Well, this just came as a very first thought, I guess I have to sleep over it - and I am definitely curious of what ideas you come up with!

[masipila]

@kaikreuzer wrote:

Hey guys, I'm now jumping on this train as well, so count me in! You've all already done amazing work (I'm especially in awe by @masipila rule logics).

Thanks, I appreciate your acknowledgement!

@kaikreuzer wrote

Let me nonetheless try to summarize what I understood what we would ideally need:

1. A mechanism to store future energy prices (and e.g. also weather forecast data)

2. A way to declare load curves for dedicated devices

3. A way to declare and/or calculate future demand of a device (i.e. heating must run 5 hours within the next 12 hours, EV must be charged to 80% by noon, dishwasher must be finished by 7am, etc.)

4. A central controller that mixes this all together and activates/controls the different devices

Adding PV systems to the solution increases the complexity by another level, since we would additionally need

5. Live information about current power consumption

6. Live information about PV power

7. Improved controller logic that considers tariff prices in combination with (cheap) PV excess power - with the difficulty, that excess power is limited, so load selection must prioritize and weigh the different loads against each other

8. Support of batteries as a "negative" load to apply when grid costs are high.

Did I miss anything crucial? Sounds already like quite some endeavor... 😲

@lsiepel wrote:

Don't want to oversimpify, but for on a high level, i think such a system should consistent of three modules;

1. Demand; device x (like a EV) demands to use a powerprofile y (like 5kw for 5 hours) with z constrains (like finishe before 8:00)

2. Provider; energy providers deliver energy according to a costprofile (like between 8:00 and 10:00 for 1 euro), your own
   PV system can be a energyprovider with a energy profile (like between 10 - 15:00 so much kwh for 0 euro) or a home battery could also be modeled as a energyprovider.

3. Engine; taking all the demands and providers into calculation according to some user set limits and preferences. (like max concurrent kw/h, timing, look for cheapest or for max self provided or whatever user defined preferences)

How I see this is as follows:

1. We need a to have a capability to store future price and forecast data The common nominator between electricity prices and different kinds of forecasts is that the data has timestamps in the future and we need to be able to store them so that we can do calculations based on them.

The common nominator for both the prices and the different forecasts is that they are future-timestamped time-series data, however the forecasted value of timestamp X may change over time as we get closer to it, but for energy optimization calculations the expected result is pretty much always that the fresh value for the same timestamp overwrites the older value for the same time.

I would like to highlight that it's not just a question of electricity prices and temperature forecasts. There are many other forecasts that can be used as an input for energy management optimizations. Solar power production forecast, cloud coverage forecast, wind power production forecast, are the first ones that come to my mind. But as said, these are nothing more than future-timestamped time-series.

2. The price of electricity is a topic of its own As @jlaur has well explained, the price of electricity consists of many different components. If we think about the European electricity market (read: Nordpool), the spot prices of the energy is just one of the components. The network transfer operator wants their share and these tariffs can be simple or hugely complex.

I recently changed our network transfer contract so that it is no longer a fixed price / kWh like it used to be. Our deal is now that

• From beginning of April until end of October we pay 4.439 c/kWh of transfer fee and electricity tax. This is easy because it's just a fixed fee which needs to be added on top of the Nordpool spot price.
• During winter time (beginning of November until end of March) the price is different during daytime and night time. During night time (22-07) the tariff is 4.439 c/kWh but during the day time it's 6.909 c/kWh. The weird thing is that during winter Sundays, the tariff is also 4.439 c/kWh days and nights.
• @jlaur is in a nice position that in Denmark they can get the prices from an API from Energi Data Service but in Finland I need to do the math by myself on top of the spot prices that I can fetch from an API.

Summa summarum: The electricity price topic has a couple of aspects:

• Fetching the data from an API
• Doing some math which may be as simple as multiplying it with a VAT multiplier and dividing by 10 to convert from EUR/MWh to c/kWh or adding a fixed tariff on top of this or it needs conditional logic that "if it's winter Saturday daytime, add this tariff"

3. Calculating the operating mode schedules for different kinds of energy consumers I just had a conversastion with @mstormi on this topic here: https://community.openhab.org/t/tibber-binding-get-best-price-items-like-awattar-binding/144223/27?u=masipila

I'm intentionally using the word "operating mode" here instead of "ON/OFF schedules" here because @mstormi has a very valid point that the world has shades of gray between the binary ON and OFF. Most ground source heat pumps have a support for SG Ready modes (there are four of them). Some simpler devices like water boilers have just two modes, ON and OFF.

I can easily see that EV charges can have other operating modes than just ON and OFF, they could for example provide an interface to allow changing the current (e.g. 8A, 10A, 13A, 16A, 32A).

Anyway, we would ideally find a concept here which is simple for users but flexible enough for different kinds of use cases. @mstormi approached this slightly differently in his solution than I did in mine, see the discussion in the link above.

But anyway, to not go into the solution yet, I'll try to write a couple of use cases in a user story format.

a) As an owner of an electric vehicle, I want to find the best time to charge the vehicle between 21 this evening and 06 tomorrow so that it will be ready in the morning. I know that I need to charge the car for X1 hours. I do not want to interrupt the charging. In other words, the X1 hours must be consecutive.

b) As an owner of a water boiler, I want to to find the best time to heat the domestic hot water tomorrow. I know that I need to heat the water for X2 hours. I do not want to interrupt the heating because stopping and re-starting it would mix the water layers. In other words, the X2 hours must be consecutive.

c) As an owner of an air-to-air heat pump, I want to optimize the heating of the house by changing between two operating modes depending on the price of electricity. Based on the weather forecast, I know that I need to have mode A enabled for X3 hours of the day and I want these to be the cheapest hours of the day. The rest of the day (all other hours) I can use mode B.

d) As an owner of a ground source heat pump that supports SG Ready modes, I want to toggle mode 4 when the price of the electricity is below a configured threshold.

(there are of course more user stories than these, but this should be a good start)

All this would fall into the "engine" category that @lsiepel was suggesting above.

[mstormi]

Some more input on levels. I also integrated EVCC Electric Vehicle Charge Controller in my EM system. They're also discussing charge modes along these lines in their project. Their current implementation is actually 4 modes:

• off
• only pv surplus (with on/off if below the minimum charge power which is 6A/1.4kW per phase)
• pv surplus but no on/off i.e. constant with a minimum of 1.4/4.2kW 1/3 phases
• max

FWIW, EV charging is pretty similar in terms of black-white-gray coloring @masipila mentioned. Some EV chargers ("wallbox" in DE) can only do on and off, but most provide a number of fixed levels (usually integer values of the amperage). Sometimes (like in Teslas) it not the charger but the car itself you need to talk to.

These EVCC modes I felt match SGr modes quite well and since I had an existing SGr implementation for heat pumps in my EMS, I just did the mapping for EVs, too.

[lsiepel]

e more input on levels. I also integrated EVCC Electric Vehicle Charge Controller in my EM system. They're also discussing charge modes along these lines in their project. Their current implementation is actually 4 modes:

• off

From a user perspective i can understand all the cases. With those cases you demonstrated that it can get really complex and without proper design this will turn into spaghetti. As allways, but specifically in this case proper seperation of concern is very important here.

As every EnergyProvider (just to be sure, this can be an external company, a battery, owned solar panels etc) knows what parameters are important (sun, cloud, windspeed, grid state, battery-wear or whatever you can think off) for it to make good prognoses for available power levels and prices it is best to keep all those details under the responsibility of this EnergyProvider. I would suggest to keep the interface between this EnergyProvider and the SchedulingEngine as simple as possible. Possibly not much more than a list of some generic propertys and a list of datapoints holding timestamp, available power, cost of power.

The same for EnergyUser / Device it knows best for itself how to calculate the PowerProfile and the interface could also be simple as: some generic properties/ constrains and the PowerProfile (i see that as a list of data points for example seconds after start, power usage)

The scheduling engine can be very lean as all the hard parts have allready been done.

I would really not suggest to make the engine responsible for determining the prices of the energy provider as that will need an extended and ever changing interface between the energyprovider and the engine.

Anyway, the above is just a suggestion, and there is one step before this and that is how all this will be tied into the openHAB achitecture.

• re-use items in some kind of group with metadata to construct a EnergyUser / EnergyProvider? Very flexible and homekit and other systems use this method allready, but how to add the additional powerprofile and specific data?
• create a new type addon ?
• allow current addons to annotate things/channels to construct a EnergyUser / EnergyProvider in a more controlled pre-set way?

[masipila]

@lsiepel can you please elaborate your thinking of the PowerProfile? For what purpose do we need it?

Not challenging, just trying to follow your thought process.

[masipila]

Regarding the considerations of EnergyUser and the existing openHAB concepts. In my eyes all these EnergyUser devices are Things. There are Bindings to most of these already and eventually Items will abstract the state changes from one mode another (like ON / OFF or from SG Ready mode 1 to 4).

The only thing that comes to my mind is that some use cases might need to know how much power the device will consume (and from which phase or all of them) if there would be some sort of a load balancing use case where you can't toggle all the devices on at the same time. Is this what you mean with the powerProfile and can you provide an example use case or user story to demonstrate the intended usage that you have in your mind?

[lsiepel]

Regarding the considerations of EnergyUser and the existing openHAB concepts. In my eyes all these EnergyUser devices are Things. There are Bindings to most of these already and eventually Items will abstract the state changes from one mode another (like ON / OFF or from SG Ready mode 1 to 4).

The only thing that comes to my mind is that some use cases might need to know how much power the device will consume (and from which phase or all of them) if there would be some sort of a load balancing use case where you can't toggle all the devices on at the same time. Is this what you mean with the powerProfile and can you provide an example use case or user story to demonstrate the intended usage that you have in your mind?

Yes, i meant the PowerProfile as a way of describing the required load usage over time. And +1 for the support of not only on/off but also different levels. Every level should have its own PowerProfile i guess.

[masipila]

Regarding the storing of the day ahead prices and other forecast data.

As I wrote in https://community.openhab.org/t/tibber-binding-get-best-price-items-like-awattar-binding/144223/40?u=masipila, the current time resolution for the day-ahead energy prices in Europe (Nordpool countries) is 1 hour. In other words, each hour has its own price.

The time resolution of the day ahead market will most likely change to 15 minutes in 2025. It is going to change to 15 minutes on the intra-day market already on 22.5.2023 but it will not yet affect consumer agreements at this point because the day-ahead market is a different animal than the intra-day market. Source (in Finnish unfortunately): https://www.fingrid.fi/sahkomarkkinat/markkinoiden-yhtenaisyys/pohjoismainen-tasehallinta/varttitase/#kysymyksia-ja-vastauksia

What does this mean in our context?

If we store the day ahead prices in different channel like awattar binding does, it would mean 4 x 24 = 96 channels (!) Awattar has currently 24 channels for today and 24 channels for tomorrow so it would mean 96 + 96 = 192 channels. I'm not convinced that this is the path we want to take since this is nothing but a time series data. It's just that some of the timestamps are in the future (rest of today and tomorrow).

So the question here is: Do we agree that openHAB core needs capability to store future-timestamped data? If yes, will we handle that in https://github.com/openhab/openhab-core/pull/3000 or in a new issue?

Once we have the capability to store future-timestamped data, we should be able to re-use that same capability to store different kinds of forecast time series data as well.

[masipila]

Regarding the concept of how to declare the future demand.

@mstormi has approached this with a nice concept which is building on top of the Smart Grid industry standard, see https://en.wikipedia.org/wiki/Smart_grid Many devices (especially heat pumps) have built-in support for Smart Grid. The concept of the smart grid is that there are 4 modes:

• Block mode: This mode can be used when the electricity is very expensive. A device can for example be turned off.
• Normal mode: The device is expected to run in normal mode.
• Low price mode: The price is cheap and the device can be told to use more energy than in the normal mode.
• Overcapacity mode: The price is very cheap (or free or the price is even negative). Device can be told to consume as much energy as it possibly can.

As @mstormi explained it in this comment and two comments below that, he has a rule that runs every full hour. It basically checks that is the hour that just started a blocked hour, normal hour, low price hour or overcapacity hour.

He has made it user-configurable that what are the "local" thresholds for how many hours of the day is to be considered "overcapacity" hours, how many should be treated as "low price" hours and how many should be "blocked" hours.

Once we have an agreed way to store the day-ahead spot prices, it's not hard to calculate the N cheapest, N most expensive etc hours from this time series. I propose that we introduce a couple of centralized algorithms to do this math and let the user to choose which one to use. This has everything to do with the consecutive hours vs. non-consecutive hours discussion that I have had with @mstormi from this comment onwards.

When (if) the time resolution changes from 1 hour to 15 minutes, I'm convinced that we need to be able to set the 4 modes so that we can force them to be consecutive.

@mstormi is currently doing this so that his solution precalculates the schedules for each mode and stores them. He then has an hourly executed Rule that updates the "mode right now" item.

It would be an elegant concept that

1. We would have the capability to store future timestamped time series data, which would be used for the day ahead spot prices (and forecasts)
2. This same capability would be used to store the result of the "planned mode" schedule
3. We would then have the "mode right now" Item.

The point here is that this concept would work for both pure spot price optomization (users without PV) and with users with PV.

Since the "planned mode" is decoupled from the "mode right now", it is possible that the PV device can update the "mode right now" item to "overcapacity" mode when this is really the case.

Edit: added the "planned mode" vs. "Right now" mode consideration

[mstormi]

Some more info for the mix: In my EMS, I'm currently using a solar forecast service that provides its info as JSON with one value per hour in the free and 15minute resolution in the paid version. So pretty much the same as the dynamic tariff handling.

We are about to move to this binding which also makes use of that service (plus another one).

[mherwege]

2. The price of electricity is a topic of its own As @jlaur has well explained, the price of electricity consists of many different components. If we think about the European electricity market (read: Nordpool), the spot prices of the energy is just one of the components. The network transfer operator wants their share and these tariffs can be simple or hugely complex.

As you are discussing this, I am just throwing in another specific calculation for network costs here, that potentially impacts the scheduling. In Belgium, we now have a capacity rate. It is defined as the maximum of the power consumption in 15 minute buckets in a month, and you get charged per kW consumption. The description is here, unfortunately only in Dutch or French. The consequence is that there is an optimum between impact of variable electricity prices and spreading the load over the day to have a lower maximum 15 minute consumption. By consequence, once a certain level has been used in a 15 minute bucket in a month, that can be used as a maximum for the rest of the month. If we think about a scheduling engine, it might actually need more flexibility to ultimately be able to insert specific logic like this.