StreamAssist icon indicating copy to clipboard operation
StreamAssist copied to clipboard

Add SIMPLE STATE sensor for sequential display of pipeline statuses

Open relust opened this issue 9 months ago • 6 comments
trafficstars

Hello. In order to better use the StreamAssit integration in the View-Assist project, it would be helpful for us to have a sensor in the StreamAssist integration that would successively publish the pipeline statuses, listening, processing and responding. These statuses are the ones used in the official satellites of Home Assistant, VoicePe and Wyoming satellite. To be able to implement my project with custom random responses and talking avatars, I need two more statuses, detected which is activated when the wake-word is detected, and finished which is activated when the tts message is actually played on the media player, an idea I took from here. @AlexxIT please look at these implementations and give me an answer.

relust avatar Jan 27 '25 20:01 relust

I think you trying to repeat built-in Assist Satellite. I plan to add it in the next update.

AlexxIT avatar Jan 29 '25 13:01 AlexxIT

Yes, I used the statuses from AssistSatellite, but I also added the FINISHED status that we have need in the automations from the View-Assist project to mark when the tts message is about to be actually played. And I would also need DETECTED or WAKE-WORD-DETECTED status for continue conversation, if custom responses are used instead of beeps, not to say for example "Yes, I'm listening" when the pipeline starts from STT-START . And I added a small delay of half a second to the LISTENING status display because it was appearing almost simultaneously with DETECTED and the DETECTED status was no longer showing

relust avatar Jan 29 '25 17:01 relust

I'll try to review your code and Satellite code to get best result.

AlexxIT avatar Jan 30 '25 09:01 AlexxIT

DETECTED is strange status. Because, as you said, you will get LISTENING status at same time. And you can add 0.5 second delay inside your component.

AlexxIT avatar Feb 08 '25 18:02 AlexxIT

If I want to use custom responses like "say please", this is ok after wake word detection but not if pipeline start at STT-START. For example, if an LLM asks me "do you want to turn on the office ligt", it should start continue conversation from STT-START without saying custm responses like "say please". This status should be called WAKE-WORD_DETECTED, but I just put it as DETECTED to be similar to the other statuses.That's why I would need both DETECTED and LISTENING statuses, for the situation when I use continue conversation. Anyway, the official Assist is also used in the ViewAssist project and I don't think I will be able to convince them to add this status and I hope they will accept the publication of events from Assist and then I will get the triggers I need from there. That's why it's up to you whether you add this status to StreamAssist. On the other hand, the status TTS-FINISHED, or simply FINISHED, would be very useful for visual effects implementation to know the moment when the tts message actually finishes playing.

relust avatar Feb 08 '25 18:02 relust

@relust check this https://github.com/AlexxIT/StreamAssist/issues/66

AlexxIT avatar Apr 24 '25 19:04 AlexxIT