openhab-webui [habassistant] Initial contribution

I would like to merge this ui, is designed to use the openHAB dialog processing capabilities from the browser.

It works over WebSocket and register an audio sink and source that can be used like any other ones, also facilitates running a single dialog processing execution using them by clicking the assistant button.

The ui is a small vue project with a couple of pages: the assistant, the local settings, and a couple of error pages. All the code related to the audio processing and the worker (which establish the websocket connection and decode the audio) is in the utils folder and it's mostly decoupled from vue, it's only connected to the assistant store to propagate the state changes.

Usage instructions are in the readme.

Let me know about any required changes or any other thing, and thank you for your time!

Signed-off-by: Miguel Álvarez [email protected]

Oct 05 '22 22:10 GiviMAD

I haven't looked at how the UI looks like and what it is able to do, but I have some comments here:

With openHAB 3, we tried to simplify the openHAB distro by reducing the choice of UIs heavily, since this was a constant complaint from users. The aim was to make the Main UI the one and only users should require for their different use cases.
We still have HABot around, which was the UI for dialog processing as well as chatting. It is supposed to be deprecated and its features should rather go to the Main UI.
The Main UI is also written in Vue, so why did you decide to implement a separate UI? Couldn't you extend the Main UI with what you think is missing in it?

If I am missing the point of your contribution with my questions, please forgive me - but those are my initial thoughts. @ghys is the master of our UIs, so I guess he has much better comments than I do.

Oct 09 '22 20:10 kaikreuzer

I actually have nothing against specialized UIs like this one which seems at first sight to be a fullscreen, minimal UI with a push button to speak and get a spoken response. It serves its purpose and could compliment the main UI without being a maintenance nightmare. Whether it would be better in the marketplace instead of the distribution itself is another debate.

However the README needs massive improvements so the casual user doesn't get false expectations: this UI is clearly not for everyone and will work only for those who have already set up their speech-to-text, interpreter, and text-to-speech engines, which is no easy task! Also some screenshots or a small GIF or the UI in action would be nice :)

Oct 10 '22 00:10 ghys

I haven't looked at how the UI looks like and what it is able to do, but I have some comments here:

With openHAB 3, we tried to simplify the openHAB distro by reducing the choice of UIs heavily, since this was a constant complaint from users. The aim was to make the Main UI the one and only users should require for their different use cases.

We still have HABot around, which was the UI for dialog processing as well as chatting. It is supposed to be deprecated and its features should rather go to the Main UI.

The Main UI is also written in Vue, so why did you decide to implement a separate UI? Couldn't you extend the Main UI with what you think is missing in it?

If I am missing the point of your contribution with my questions, please forgive me - but those are my initial thoughts. @ghys is the master of our UIs, so I guess he has much better comments than I do.

My initial motivation for my contribution is that this is very useful for me at this moment because I already have some voice commands implemented in openHAB but not speakers connected to it in all the rooms, so I wanted something to quickly use my mobile or laptop to speak to openHAB in some situations.

I didn't though on adding it to HABot because I didn't see any relation between the dialog capabilities of the core and it other than the openNLP interpreter so I though they were not really related, never used it too much so probably I'm wrong.

I though it was ok as a separate interface because of this points:

The interface is mostly a single page and I always like the concept of the alternative interfaces, not to replace functionalities of the main one like is happening in some cases, but to plugin alternative functionalities.
A lot of people is using the integration with the google and amazon solutions, and I think a lot of them will not get interested on connect their own speakers/mics to openHAB. Right now you need to have a speaker and microphone connected to your server to use javasound sink/source or rely on the pulseaudiobinding sink/source but I don't think there are more alternatives to try the DialogProcessor capabilities, this addon looks to me like something useful for people how wants to give a try to the voice processing capabilities without extra hardware or hesitation but still I think many people will not use it so I saw it right like something optional.
The openHAB core is not using websockets for anything, so I though that if I want to use them, better to do it in some separate addon.

I like the WebSocket "subprotocol" implemented, I think is a good idea because right now keyword spotting for the speaker I have in the living room is happening on the openHAB server, and I don't think that is reliable if I want like 6 speakers (my openhab instance runs on a small arm64 server with like other 10 applications). My idea was that if this PR gets accepted in the future I can write a small executable that allows me to connect more speakers to openHAB while running the keyword spotting on those and rely on the single dialog execution capability of the core using this websocket. So I saw this alternative interface also like a way of having an optional audio protocol over WebSocket enabled on the server.

Let me know if you have further suggestions after reading this, thank you for the feedback!

Oct 10 '22 00:10 GiviMAD

However the README needs massive improvements so the casual user doesn't get false expectations: this UI is clearly not for everyone and will work only for those who have already set up their speech-to-text, interpreter, and text-to-speech engines, which is no easy task! Also some screenshots or a small GIF or the UI in action would be nice :)

Sure, I will do it, I'll also highlight other limitations at the beginning.

Oct 10 '22 00:10 GiviMAD

@ghys I have updated the readme, let me know if you think there something else that should be highlighted or explained in more detail.

Oct 12 '22 10:10 GiviMAD

Thanks for sharing your reasoning, @GiviMAD. While I'd like to stick to the path to not (again) end up in a plethora of user interfaces for openHAB, I won't veto a merge, if @ghys thinks it makes sense to add here. His idea of putting it to the marketplace instead of the distro might be worth a thought, though.

Another thing to contemplate about is the name: "habassistant" very much resembles "home assistant" and I wouldn't want to give any impression that we are trying to "steal" their name or not respect it by choosing something very similar here. Maybe you have a good idea for an alternative name of the UI?

Oct 15 '22 20:10 kaikreuzer

His idea of putting it to the marketplace instead of the distro might be worth a thought, though.

I don't understand what this means, will this involve moving the package to another repository?

Another thing to contemplate about is the name: "habassistant" very much resembles "home assistant" and I wouldn't want to give any impression that we are trying to "steal" their name or not respect it by choosing something very similar here. Maybe you have a good idea for an alternative name of the UI?

Yes they are pretty similar indeed, I didn't realize it. I've been thinking about another one and I like 'HAB Speaker', @kaikreuzer, @ghys do you think this is a better one?

Thank you for your time and feedback!

Oct 16 '22 09:10 GiviMAD

After thinking about it, maybe this fits more under the category of 'misc' addons than in the 'alternative ui' one. I made it in this repo because I want it to be shown on the ui list like the HABot (now I understand that can be accomplish from any kind of add-on) and because I used that project as model for adding the new bundle, but indeed it's not an alternative ui. Just a thought, let me know what you think is the best option and I'll stick to it. And again thank you for your time!

Oct 16 '22 10:10 GiviMAD

I'll try to add this to the marketplace, I'm suppose it what have more sense in case it ends up been poorly used. @kaikreuzer please let me know if you prefer that I use a different name that HABSpeaker. Thank you for the feedback!

Oct 25 '22 17:10 GiviMAD

Thanks @GiviMAD and sorry for the late reply. HABSpeaker sounds good to me, I guess it describes its purpose nicely. I think the marketplace is a good way to start. If it becomes very popular, we can still think about adding it to the distro itself. If I find the time, I'm myself actually very interested in trying out all your work around voice processing - so far, I didn't manage it. 😢 Keep up the good work!

Nov 02 '22 20:11 kaikreuzer