Add EventWebSocket

[J-N-K]

It has been reported in the past that high-traffic applications like HABApp and systems with a large number of subscribers are affected by performance limits in the REST/SSE implementation.

This PR adds a websocket connection that can be used to receive events on the event bus (filtered by type and source) and accepts ItemStateEvents and ItemCommandEvents from the remote side. The connection is protected by a heartbeat-protocol and can be easily integrated in UIs as web sockets are available in ECMA script. Access is protected by token or basic auth.

@spacemanspiff2007 has done some performance tests:

REST + SSE (Basic Auth)

Sent: 8894 msgs
Dur : 10.0 sec
      889.3 msg/sec

REST + SSE (Token)

Sent: 2532 msgs
Dur : 10.0 sec
      253.2 msg/sec

Websockets

Sent: 28418 msgs
Dur : 10.0 sec
      2841.6 msg/sec

So compared to using the REST API and SSE with token-based auth around 10x better performance can be achieved.

[splatch]

Given amount of troubles with SSE (proxy handling, UI disconnects, age and possible sunsetting of it), I think that making websockets a primary event handling mechanism for web is important discussion to be made. I am really looking forward to get it introduced and hopefully utilized by UI at some point too, especially that it allows wider and more customized payloads as well as bidirectional communication between participants.

One note - AFAIR part of performance boost might be coming from fact that websocket connection, once launched, does not validate token nor cookie as it is separate network channel. Token you validate at the time of establishing of connection might expire or be cancelled through logout but websocket connection will stay unaware of that fact unless it is shut by server.

[spacemanspiff2007]

One note - AFAIR part of performance boost might be coming from fact that websocket connection, once launched, does not validate token nor cookie as it is separate network channel. Token you validate at the time of establishing of connection might expire or be cancelled through logout but websocket connection will stay unaware of that fact unless it is shut by server.

With basic auth there is a cache that saves the result of the validation. So the perfomance should be the same (since it's only a lookup) but it's still factor 3 slower than websockets. The cache is what gives basic auth a 4x performance boost over token.

Currently authentication is not even required for all rest endpoints so I am not sure if it makes sense to already implement corner cases in the websocket connection. However if required it should be a fairly easy thing to solve (take it with a grain of salt since I am not a java programmer): Attach user and token to the websocket connection. If one of both gets deleted/expires then just close the connection.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/habapp-potential-thread-event-contention-issue/135645/22

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/habapp-potential-thread-event-contention-issue/135645/23

[leifclaesson]

I for one would love to see this feature implemented. More efficient event handling would make a big difference running openHAB on popular low power platforms like Raspberry Pi.

[J-N-K]

done

[spacemanspiff2007]

Just a small note since I am really excited since this seems to get traction.

Would it be possible to automatically resolve the json strings in json to the proper json structure?

e.g. this

{
  "type": "ClassType"
  "payload": "{\"a\": 1}"
}

to

{
  "type": "ClassType"
  "payload": {
    "a": 1
  }
}

Currently I go recursively through all the entries and check if it starts/ends with a [] or a {} pair and then try to load the entry as a json. However it would be very nice from an api consumer perspective if that would be implemented on the server side.

[J-N-K]

I understand your intention but since the payload is a String in openHAB, I would prefer to keep it like that. The reason is that I see the websocket as a transparent extension to the event bus and therefore no processing should occur.

[spacemanspiff2007]

But as far as I understand on the event bus you have the corresponding event class instances and not only the string values. The closest websocket equivalent would be the json representation and not the string representation so imho this would be an argument for the deserialization of values.

---

To get a better understanding - where do you see the benefit in viewing it as an "extension" and not an "external API"? There are a view places where consuming the RestAPI is unnecessarily hard and logic that should be core logic is offloaded to the api consumer and I can imagine the reason is that the api is viewed more like an "extension".

I'd love to see a mindset shift where the development target is "the api is there to make interaction with openHAB easy" and not "it's the same like in java but with REST".

I get that it's much easier and less code in the core if it's an "extension" and maybe I'm missing some arguments so I'm curios what other reasons might be.

Nonetheless I think web sockets are a huge step forward from sse and I'm really looking forward to get this merged!

[J-N-K]

According to the developer documentation the payload can contain "any string representation", not necessarily JSON (even though all core events do so). What should happen when the payload does not contain a JSON but e.g. a plain String, an XML or YAML? The consumer might know if he wants to process that event, but the websocket classes can't properly detect that.

[spacemanspiff2007]

What should happen when the payload does not contain a JSON but e.g. a plain String, an XML or YAML? The consumer might know if he wants to process that event, but the websocket classes can't properly detect that.

I would have taken the pragmatic approach and if the json deserialization fails just pass the original string. That way it's possible to provide convenience while still maintaining correct behavior.

Pseudocode:

try {
   payload = deserialize_json(string_payload);
}
catch (Exception) {
    payload = string_payload;
}

Another option would be to to deserialize only the core events since for these events it's defined that the string is serialized as json.

The third option would be to define that the serialization for events has to be json. It's the de facto standard now so why not make it mandatory for all events.

[J-N-K]

Do you know of any documentation for SSE besides this sentence? IMO it would be best to put the docs for web sockets close to that. Or maybe at the end of [this (https://www.openhab.org/docs/developer/utils/events.html)?

[wborn]

IMO it would be best to put the docs for web sockets close to that.

The "restdocs.html" seems to be only about the REST API while this is a WebSocket API. Perhaps a new page should be added for this? E.g. with some links in the Event Bus docs telling you can also subscribe to them using the REST/WebSocket API. WDYT @Confectrician?

I also found the REST API page has some subscription examples (which will not work because smarthome got renamed to openhab :roll_eyes:, https://github.com/openhab/openhab-docs/pull/1951 ):

[Confectrician]

Adding a new page sounds good.

[J-N-K]

While adding the documentation I found that the events for configuring the websocket use a preceding slash (/xyz) for the topic while openHAB usually omit that (openhab/xyz). Do you think this should be changed? I also found a bug with Basic Authentication, so I have to prepare a PR anyway.

[wborn]

Do you think this should be changed?

The openhab prefix might help to differentiate between topics if you create some third party addon. It might also be a bit easier to switch between SSE and WS when the topics are the same.

[J-N-K]

My concern was more about the leading /, not the topic itself. I guess the topic does not matter that much because the events are only exchanged between server/client on the connection.

[Confectrician]

The ability of omitting the / also for the websocket connection would be a consistent behavior across the different available apis, so i personally would go for this approach.