openhab-core
openhab-core copied to clipboard
openhab
Add `getLastChange`, `getLastUpdate`, and `getPreviousState` to GenericItem
These methods are handy in many situations and isn't possible with mapdb persistence, so previously it would require something like influxdb.
It also solves the issue of needing to wait until persistence write operation is finished (if it was at all set up to persist on change), so this gives us a cheap and reliable / dependable way of getting the last change / last update time.
See discussions in https://community.openhab.org/t/persistence-on-off-transitions-with-influx/156271/ https://community.openhab.org/t/blockly-persistence-last-changed-returns-no-result-quite-often/157842
I like the idea of this information being part of the Item. Of course it won't go back beyond OH startup but something is better than nothing. It certainly handles this use car better than persistence I think given all the different ways perspective can be configured.
How does this interact with restoreOnStartup? If an item is restored and not changed does this return the restored time or null? What makes the most sense? I'm not sure I know (it's close to bed time here so maybe I'll sleep on it).
How does this interact with restoreOnStartup? If an item is restored and not changed does this return the restored time or null?
It is not aware of persistence restoration, so I guess once persistence restores the state, it will return the time the state was restored.
Imho it should be possible to query these values from the db so restore on startup should work as expected. Will these fields also be part of the RestAPI / Websockets response?
Imho it should be possible to query these values from the db so restore on startup should work as expected.
@spacemanspiff2007 would you mind elaborating this please?
Will these fields also be part of the RestAPI / Websockets response?
Good point. I will add them there too.
It's possible to query the last persisted state from persistence and subsequently restore this state as the current state on startup. It's also possible to query the persisted state before that. Subsequently it should be possible (imho) to get all the values from the persistence service so restore on startup should also work for these fields as expected.
It's possible to query the last persisted state from persistence and subsequently restore this state as the current state on startup.
Yes, isn't this what restoreOnStartup does?
It's also possible to query the persisted state before that. Subsequently it should be possible (imho) to get all the values from the persistence service so restore on startup should also work for these fields as expected.
I don't quite understand the above statements.
This PR doesn't interfere with persistence's restoreOnStartup. If I understand @rkoshak's question correctly, it is a question of whether getPreviousState should return (java) null as is the case when there was no previous state being set on the item, or whether it should return UnDefType.NULL because that's the item's state before restoration. Currently it's the latter.
I think it's a misunderstanding.
All I am saying is that it's technically possible to restore these new fields from persistence so imho it makes sense to do so during restoreOnStartup.
@spacemanspiff2007 I've added it to the REST response
{
"link": "http://192.168.1.10:8484/rest/items/TestSwitch1",
"state": "ON",
"previousState": "NULL",
"lastUpdate": 1723872081231,
"lastChange": 1723871965049,
"editable": false,
"type": "Switch",
"name": "TestSwitch1",
"tags": [],
"groupNames": [
"TestSwitches"
]
}
All I am saying is that it's technically possible to restore these new fields from persistence so imho it makes sense to do so during
restoreOnStartup.
That depends on the persistence service. And it is not the case for mapDB, the most used persistence service for restoreOnStartup.
I would argue that, when the item state is updated as a consequence of restoreOnStartup, it should not update these lastChange and lastUpdate fields because now is probably the wrong time. But I am not sure this is feasible. I doubt the item knows anything about the source of the update/change.
In general, I think having the extra fields is a good idea. I just feel like we are still not there to give an answer to a simple question: when did my item last change? If I use this new function (lastChange):
- Item did changed since startup -> OK, correct answer, and better then what I could get from persistence as persistence could miss the everyChange strategy or could have compressed
- Item did not change since start -> now, which is wrong
- If I can figure out the answer is wrong and I have a persistence service with everyChange: call lastChange on the service -> a. Current state equal to last state in DB -> correct answer from persistence (ignoring for now impact of compression) b. Current state different from last state in DB -> no answer (null) c. If no answer, you could return now in the calling rule, although in this case the startup time might be a more appropriate approximation
In an ideal world, mapDB would just store the lastChange time of an item with the state and we could restore that together with the state. I am just not sure it is easily feasible. Thinking out loud here: As mapDB is kind of a special case which is just used for restoreOnStartup, would it make sense to remove restoreOnStartup from persistence all together and make it part of the core infrastructure, just to take care of this? I would argue you should then persist all items in mapDB all the time, invisible to the user configuration, and use only that special DB in startup to restore item states and their accompanying update dates.
We can keep adding layers to this, but there are always issues and exceptions. I like the idea of keeping lastChange and lastUpdate with the item. But I think the real simplification comes when we have a simpler restore mechanism on startup. Because than, you always get the right answer. Other persistence services then have the focus on data for graphing and calculations, which makes sense in my view.
OK, I think I now understand what @spacemanspiff2007 was alluding to, thanks to @mherwege's explanation.
Thinking out loud here: As mapDB is kind of a special case which is just used for restoreOnStartup, would it make sense to remove restoreOnStartup from persistence all together and make it part of the core infrastructure, just to take care of this? I would argue you should then persist all items in mapDB all the time, invisible to the user configuration, and use only that special DB in startup to restore item states and their accompanying update dates.
Not a bad idea.
In an ideal world, mapDB would just store the lastChange time of an item with the state and we could restore that together with the state. I am just not sure it is easily feasible.
Why would this not be feasible? I beleive all the apis already exists in persistence to get the time from persistence. Once you have that it's just a matter of adding the last change and last update time as a parameter on the Item.
Thinking out loud here: As mapDB is kind of a special case which is just used for restoreOnStartup,
I'm not sure we can make that case that's it's always only used for restoreOnStartup. It's useable if you only care about the most recent change to the Item in a rule (given it's current default strategies). One could change the strategies and it would work for most recent update.
But if these properties are added to the Item and we go down a path where those are made available and accurate on the the Item this use case is handled.
I would argue you should then persist all items in mapDB all the time, invisible to the user configuration, and use only that special DB in startup to restore item states and their accompanying update dates.
Be careful here.
Not all uses want all Items restored.
Not all users want all Items restored to their most recent change or update (e.g they use a rule with .persist to control when the state gets saved).
These use cases are now currently supported. Any move of restoreOnStartup to core would need to handle these use cases too. How the Item's state is saved and which Items are restored needs to be configurable on an Item by Item basis like it is now in persistence.
That depends on the persistence service. And it is not the case for mapDB, the most used persistence service for restoreOnStartup.
It would be trivial to modify mapDB persistence service to save the additional three values. A quick glance at the docs shows it should be possible. If not it's always possible to name mangle additional values with a separator that's not a valid item name (e.g. MyItem#lastUpdate)
The question is - as you correctly pointed out - if that's something that results in a meaningful behavior.
2. Item did not change since start -> now, which is wrong
No - it should return null because it's undefined.
These use cases are now currently supported. Any move of restoreOnStartup to core would need to handle these use cases too. How the Item's state is saved and which Items are restored needs to be configurable on an Item by Item basis like it is now in persistence.
I strongly agree. For many devices that report the state themselves I explicitly do not save the item state so I don't have the wrong state after startup. If the restore makes sense depends strongly on the device, the user and how the rules are written. I for once was struggling with a rule and strange startup behavior during testing and the root cause was rrd4j restoring states during startup which was automatically configured (rrd4j).
Just a hint:
HABApp already provides the last_update and last_change datetimes. I deliberately chose not to make them null to simplify the rules so the users don't have to always check for that. Instead I chose to initialize them with the timestamp when the item gets created.
That would also result in a proper behavior when restoring from persistence.
Time 0 - item creation:
state: NULL
previousState: NULL
lastUpdate: 0,
lastChange: 0,
...
Time 10 - persistence restore:
state: "asdf"
previousState: NULL
lastUpdate: 10,
lastChange: 10,
So I am basically suggesting the opposite of what I wrote above :see_no_evil: :rofl:
These use cases are now currently supported. Any move of restoreOnStartup to core would need to handle these use cases too. How the Item's state is saved and which Items are restored needs to be configurable on an Item by Item basis like it is now in persistence.
Each item can have a special metadata restoreOnStartup=true. When this is on a group item, persist all its direct members.
Each item can have a special metadata
restoreOnStartup=true. When this is on a group item, persist all its direct members.
I think, what @rkoshak also implied is that one may want to use different rules for persisting values, not just on every change, but e.g. persist it once and always use that same value at startup. And that would mean we also need to support that part of the configuration in core. And then we are back to doing it the way it is done now anyway.
Thinking further about it:
- Changing the
restoreOnStartuplogic to also restorelastChange,lastUpdateandpreviousStateat startup is fairly easy to do. The method to change is https://github.com/openhab/openhab-core/blob/506ccd4f4db207f97abe2672a5ee30859b635426/bundles/org.openhab.core.persistence/src/main/java/org/openhab/core/persistence/internal/PersistenceManagerImpl.java#L521. You may need some extra methods (or an extendedsetStatemethod that has extra parameters) inGenericItemto also set these fields, although it should probably only be used forrestoreOnStartup. - Many persistence services can already provide the extra info required. MapDB cannot and should be extended to store and retrieve the extra information. I don't know how easy that would be to do. We also seem to be using a very old version of MapDB, so I don't know how much impact that has on feasability.
With the above logic in place, it also would make sense to revisit the persistence extension actions to not return null, but return what is actually in the database, and document it as a breaking change, advising users to use the Item methods for this use case.
- Many persistence services can already provide the extra info required. MapDB cannot and should be extended to store and retrieve the extra information. I don't know how easy that would be to do. We also seem to be using a very old version of MapDB, so I don't know how much impact that has on feasability.
Looking at MapDB code, it is actually not that difficult. The item state and time are already stored as a JSON string. Adding extra fields should not be a problem.
I think, what @rkoshak also implied is that one may want to use different rules for persisting values, not just on every change, but e.g. persist it once and always use that same value at startup. And that would mean we also need to support that part of the configuration in core. And then we are back to doing it the way it is done now anyway.
Since a user needs a rule for calling .persist, if there's also way to turn off the restoreOnStartup for that Item in this new core feature, maybe that would be sufficient as they could create a system started rule to restore the Item's state manually from whatever persistence they are using instead of this new core one. Or maybe we can add a persistence extension to restore the state with the most recent value in the database that could be called from a rule in place of the restoreOnStartup strategy that would be removed.
This new core service doesn't need to do all the use cases. We just need to ensure that all the use cases are covered somehow.
With https://github.com/openhab/openhab-core/pull/4324?notification_referrer_id=NT_kwDOAGFtyLMxMTU1MDk4MDI0MDo2Mzg1MDk2#issuecomment-2286378895 I think we will reliably be able to count on persistence being there by runlevel 100. So maybe a system started rule with previousState would be sufficient. I don't know though, I haven't fully explored all the edge cases that can crop up due to timing and such.
- Item did not change since start -> now, which is wrong
No - it should return null because it's undefined.
Maybe we are thinking too hard about this.
What do we intend these new properties to represent? I see two options:
- reflect the changes/updates made to the Item
- refelct the changes/updates to the device
If it's just 1, we don't really need to worry about most of these complexities. If an Item doesn't change after OH startup, getLastChange will be the time when the Item was created and set from restoreOnStartup and getPreviousState would be NULL. If you want the actual device information and not just information about the Item during this run session of OH, then you need to use the existing persistence capabilities. But this also means we don't need a new restoreOnStartup service in core or any of that complexity and the implementation becomes pretty simple.
If we want to make 2 happen then it gets much more complicated, as we can see and we probably need something built into core.
But do we really need to solve 2 here? Maybe 1 is sufficient? I think users who prefer not to use persistence at all would love to have 1.
We also need to thing of second tier impacts like the increase in writes for those running on SD cards and the like if this core restoreOnStartup feature becomes the default behavior.
So what's the next step?
- Changing the
restoreOnStartuplogic to also restorelastChange,lastUpdateandpreviousStateat startup is fairly easy to do. The method to change is https://github.com/openhab/openhab-core/blob/506ccd4f4db207f97abe2672a5ee30859b635426/bundles/org.openhab.core.persistence/src/main/java/org/openhab/core/persistence/internal/PersistenceManagerImpl.java#L521 . You may need some extra methods (or an extendedsetStatemethod that has extra parameters) inGenericItemto also set these fields, although it should probably only be used forrestoreOnStartup.
If we are going to proceed in this direction, I would suggest:
- First merge this PR without the extra
setState/ setters for lastChange, lastUpdate etc. - If and when the integration with the persistence extensions is added, then add the setters as you suggested. This way, it's clearer how the setters are implemented according to how it's actually used.
If we are going to proceed in this direction, I would suggest:
- First merge this PR without the extra
setState/ setters for lastChange, lastUpdate etc.- If and when the integration with the persistence extensions is added, then add the setters as you suggested. This way, it's clearer how the setters are implemented according to how it's actually used.
I am fine with this.
I think this is the best way, too.
Maybe it makes sense to make persistence of lastChange etc. configurable. This could and should be discussed in the according PR.
Do the the new values also get reported in the corresponding item events?
This pull request has been mentioned on openHAB Community. There might be relevant details there:
https://community.openhab.org/t/lastupdate-and-null-values/158134/6
This pull request has been mentioned on openHAB Community. There might be relevant details there:
https://community.openhab.org/t/how-to-monitor-if-item-is-frequently-updated/158260/6
how do we resolve the access to these methods vs the persistence methods with the same name within rulesDSL?
how do we resolve the access to these methods vs the persistence methods with the same name within rulesDSL?
Maybe I miss something here, but what gets called would be determined by the imports I would think. As both items and persistence are imported by default, it indeed leads to a conflict which will force changing the rules to have a fully qualified name.
The options then become:
- Give these new methods another name: this is fully backward compatible.
- Change the name of the existing persistence actions to
getLastPersistedState,getLastPersistedUpdateandgetPreviousPersistedState.
In both cases, rules would work without being changed, but 2 would have a slight change in behaviour. I would still prefer 2 because it is more natural not to call persistence if not needed. If one absolutely wants the value from persistence the rule would need to be changed, but it would be very clear what is expected. At the same time, the persistence actions could be modified again to not return null, but the last effective persisted value. 2 also requires adapting the scripting languages to call these changed methods.
2. Change the name of the existing persistence actions to
getLastPersistedState,getLastPersistedUpdateandgetPreviousPersistedState.
I think I like this option too!
Should you not rather have these methods on the Channels rather than the Items? In that case one would be able to create respective Items with (say) a profile that reads the Channel's getLastUpdated (say) property.
Should you not rather have these methods on the Channels rather than the Items? In that case one would be able to create respective Items with (say) a profile that reads the Channel's getLastUpdated (say) property.
That would make it impossible to restore these values from persistence at startup. Items have a state, channels only pass on a state. I don´t think we should do this.
If you insist on doing it at Item level, then I think you need to find a mechanism whereby the new values from these functions can be accessed in the UI. (e.g. via a profile). Otherwise they are only accessible via rules..
a mechanism whereby the new values from these functions can be accessed in the UI
They are available through the REST API and it is already a part of this PR.