Add timestamp to Events

[jbaudisch]

trafficstars

Current JSON for events looks i.E. like this:

{
    'topic': 'openhab/items/{ITEM_NAME}/statechanged',
    'payload': '{
        "type": "OnOff",
        "value": "OFF",
        "oldType": "OnOff",
        "oldValue": "ON"
    }',
    'type': 'ItemStateChangedEvent'
 }

Would be nice to also have a timestamp attached to it like:

{
    'topic': 'openhab/items/{ITEM_NAME}/statechanged',
    'payload': '{
        'type": 'OnOff',
        'value": 'OFF',
        'oldType': 'OnOff',
        'oldValue': 'ON'
    }',
    'type': 'ItemStateChangedEvent',
    'timestamp': '2022-07-08T10:36:28.189830'    # (or another format)
 }

Greetings,

[J-N-K]

Also see the discussion in #3000

[kaikreuzer]

@J-N-K I guess this issue here would be simpler to address than #3000, because it is actually not about adding a timestamp to the state, but adding a timestamp of the state changed event (imho something slightly different).

Considering the title, it then would indeed make sense to add a timestamp to ANY event that we are sending. But I am not quite sure what's the real value of it. We do not send timestamps with REST responses and the events should be considered to be "streamed REST responses", that's why I'd try to keep them similar. A subscribed client should receive them in near-realtime, so it can use its own system time, if it needs to associate a time with it.

In general, I always tried to avoid introducing timestamps around item states specifically. The main reason for it is that I think they suggest a precision, which is not there. After all, the items are just a digital twin of the real world - and by adding timestamps, people have the impression that those reflect the moment of change in the real world. But this is not the case; we do not know how long the binding takes to recognize the change. We do not know how long the core framework takes to receive the info and update the item itself and how long it then takes to construct and send an event for it. That's why I'm usually a bit cautious on that matter.

[jbaudisch]

"A subscribed client should receive them in near-realtime, so it can use its own system time [...]"

@kaikreuzer You are right by saying that one can use the system time, I do that in my current project. But in my case, sometimes this time is different (>1s) to the "real" (openhab's) time (e.g. through a lag or whatever) and in my setup, this 1s inaccuracy is dramatic.

"[...] reflect the moment of change in the real world [...]"

I see your point here and agree with it, but couldn't this misleading interpretation be avoided by the name of the field? In example "sys_time" or whatever, instead of "timestamp"? I do not care about the fact, that the timestamp is a representation of the real world, but in fact, I do care about the time passes between two events and this should be as precise as possible (Yes, I could add an extra item for that, but I would've do this for every and each item existing, and for me, this is to much overhead for such a simple task).

[kaikreuzer]

@jbaudisch Yes, you are right, that's why I said that this request here is slightly different (and hence easier) than #3000, which deals with item state timestamps.

So as we here talk about system timestamps of event creation, this should be all fine and my concerns do not apply here. What remains is this remark, though:

But I am not quite sure what's the real value of it. We do not send timestamps with REST responses and the events should be considered to be "streamed REST responses", that's why I'd try to keep them similar.

So if you could maybe elaborate a bit on your use case and why you think timestamps for events would be helpful, it would help me understand - thanks!

[spacemanspiff2007]

@jbaudisch What exactly are you interested in? The date when the event was emitted or the date when the item updated the state?

Since you are using the jsons I'm assuming you are consuming the sse events over the rest api. Is that correct? Maybe it would be enough to add this to

• #2956

[jbaudisch]

@kaikreuzer @spacemanspiff2007

Okay, let me define my use-case.

I write a research software which learns user-activity based on sensor-events and their time interval. Based on openHAB's REST API, I consuming the sse events and process only "ItemStateChangedEvent". So for simplicity, lets consider this scenario: The system learns the activity "walking from bathroom to bedroom". For this, it learns that the bathroom motion switch and bedroom motion switch turn on within 4 seconds. Currently, it measures the time when the event arrives for that time calculation. Now, if the sse is delayed due to a network failure or something, I get a delay of maybe 1 second (so 5 seconds in total). Then the system doesn't recognize the activity (4<5). So the idea now was to outsource this timestamp to openHAB and just read it from the sse-data to avoid this.

[digitaldan]

@jbaudisch as an alternative if SSE ends up not working for you,. would using a openHAB persistence strategy maybe make this easier? So persisting all item events to MySQL, PostgreSQL, InfluxDB, etc... Then directly access the DB and analyze data points? I have often though about what kind of trend analysis could be done on large amounts of historic data like that.

[spacemanspiff2007]

That's a great idea @digitaldan. Since the data has to be stored anyway why not use a DB from which the data can also be directly consumed from.

Another option would be to run the see listener directly on your openHAB device and create the timestamps there. That way there will be no network delay. I'm getting ~30ms round trip times for my HABApp installation (HABApp -> openHAB -> HABApp). You could try that and push events out from HABApp (e.g. over mqtt, so openHAB -> HABApp -> mqtt) just as a quick fix to see if network issues are really your problem.

Some personal notes: I've had mixed experience with the reaction time of my ZWave movement sensors. Most of the time they are very snappy but sometimes they take noticeably longer to react. So it might be that the introduced delay is not because of the SEE connection but on a sensor level (reaction time + transmission time + binding processing time).

[kaikreuzer]

@jbaudisch I would agree with @digitaldan and @spacemanspiff2007 that persisting the events sounds like the best fit for your usecase - your logic would then be able to very flexibly query the database with all events. Wdyt about it?

[jbaudisch]

@digitaldan, @spacemanspiff2007, @kaikreuzer Thank you for your input and sorry for my late response. I think outsourcing the persistence of the event-part to openHAB would fit my needs and I'll try to integrate it in my software.

Thanks alot!

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/openhab-4-0-wishlist/142388/449