StreamAssist icon indicating copy to clipboard operation
StreamAssist copied to clipboard

Introducing a new project as an alternative to StreamAssist

Open AlexxIT opened this issue 8 months ago • 11 comments

You may know that I have a powerful streaming project called go2rtc. I recently added support for the wyoming protocol to it. This will allow for the creation of powerful and flexible voice satellites.

Pros:

  • A large number of options for receiving and for playing back audio streams
  • The audio stream processing and wake word dector can be moved to go2rtc, this will significantly reduce the load on the Home Assistant
  • Scripting engine, will allow you to add some customization

Cons:

  • Perhaps the first setup will be a little more complicated than StreamAssist

You can read more in the readme.

For now, the update is only available in dev version, but you can already try it out.

The implementation turned out to be quite complex, so not everything may not work stably. Waiting for your feedback and wishes.

AlexxIT avatar Apr 24 '25 19:04 AlexxIT

unclear how to install dev HA addon. Hosting in different addon repo? Really want to test this out.

Scags104 avatar May 20 '25 13:05 Scags104

All versions of the addon are in the same repository.

AlexxIT avatar May 21 '25 09:05 AlexxIT

Hey! @AlexxIT i am running the go2rtc (master) which i guess is the latest unstable version, but i am struggling to find a way to integrate it with openwakeword like stream assist. Any hints would be greatly appreciated! Thank you !

nioakeim avatar Aug 11 '25 07:08 nioakeim

There are examples in the links above.

AlexxIT avatar Aug 13 '25 13:08 AlexxIT

Hi @AlexxIT , it looks very promising. I would like to give it try but i'm on frigate hassio addon. Embedded go2RTC binary is version 1.9.9 (so no wyoming here), but i think that also additional ports (2 per satellite, mic+snd) would need to be forwarded from the container to the host in order for it to work, correct? I was thinking of replacing StreamAssist this way, I'm experiencing some memory leaks recently (as described here: https://github.com/AlexxIT/StreamAssist/issues/63, but i'm on a 12 core / 16GB mini pc) as soon as i activate a MIC in any of the StreamAssist entries.

override80 avatar Aug 18 '25 09:08 override80

You don't need to open ports if you using addon. All containers and HA in same closed docker network.

https://github.com/AlexxIT/WebRTC/blob/cd84c4baf1f24cafcef2a7e73a8d65a79a14cddf/custom_components/webrtc/config_flow.py#L50-L56

AlexxIT avatar Aug 18 '25 12:08 AlexxIT

@AlexxIT This is amazing! And it's working very fine!

I came up with these settings, in case somebody wants another example. In my specific case, I use a speaker named media_player.jam. The stream is named "studio" from an hikvision-compatible camera mic.

go2rtc:
  log:
    wyoming: trace
  wyoming:
    studio:
      listen: :10700
      name: Studio Satellite
      wake_uri: tcp://192.168.1.98:10400?name=alexa_v0.1&name=hey_jarvis_v0.1&name=hey_mycroft_v0.1&name=hey_rhasspy_v0.1
      vad_threshold: 0.5
      event:
        internal-detection: |
          let token = 'REPLACE_WITH_HA_LONG_LIVED_TOKEN';
          fetch('http://192.168.1.98:8123/api/services/media_player/play_media', {
            method: 'POST',
            headers: {
              'Authorization': 'Bearer '+token,
              'Content-Type': 'application/json'
            },
            body: toJSON({
              entity_id: 'media_player.jam',
              media_content_id: 'http://192.168.1.98:8123/local/beep.wav',
              media_content_type: 'audio/x-wav',
            }),
          }).ok;
          WriteEvent("run-pipeline", '{"start_stage":"asr","end_stage":"tts"}') && Stream()
        synthesize: |
          let text = fromJSON(Data).text;
          let token = 'REPLACE_WITH_HA_LONG_LIVED_TOKEN';
          fetch('http://192.168.1.98:8123/api/services/tts/speak', {
            method: 'POST',
            headers: {
              'Authorization': 'Bearer '+token,
              'Content-Type': 'application/json'
            },
            body: toJSON({
              entity_id: 'tts.google_ai_tts',
              media_player_entity_id: 'media_player.jam',
              message: text,
              cache: true,
            }),
          }).ok
        audio-stop: |
          let timestamp = fromJSON(Data).timestamp;
          let delay = string(timestamp)+'s';
          Sleep(delay) && WriteEvent("played") && Detect()
  streams:
      studio:
        - rtsp://USER:[email protected]:554/Streaming/Channels/101?video&audio

Only thing i noticed is that i need to reload the wyoming satellite in the Integrations - Wyoming page whenever something changes on goRTC in Frigate.

To successfully run it in frigate container, i just overwrote the go2rtc binary in the frigate container in /usr/local/go2rtc/bin/go2rtc with the nightly built one:

docker cp go2rtc addon_ccab4aaf_frigate-fa:/root
docker exec -it addon_ccab4aaf_frigate-fa bash
pgrep go2rtc | xargs kill ; cp /root/g2rtc /usr/local/go2rtc/bin/

This will trigger a restart of the go2rtc process, the service is supervised via s6supervise.


EDIT: Well it turns out i have to reload the integration pretty often to get it working consistenly, every time i issue a command

EDIT2: Turns out the timestamp in Data json is ms, not s, so: let delay = string(timestamp)+'ms';

override80 avatar Aug 19 '25 10:08 override80

@AlexxIT is there any reason why the audio-stop event is not executed? Sometimes it just stops at the the synthesize expr:

info | 2025-08-19 18:48:17 | wyoming | event=internal-detection data={"name":"hey_jarvis_v0.1"} payload size=0
info | 2025-08-19 18:48:17 | wyoming | event=internal-detection expr result=true
info | 2025-08-19 18:48:17 | wyoming | event=transcribe data={"language": "it-IT"} payload size=0
info | 2025-08-19 18:48:19 | wyoming | event=voice-started data={"timestamp": 2190} payload size=0
info | 2025-08-19 18:48:21 | wyoming | event=voice-stopped data={"timestamp": 4080} payload size=0
info | 2025-08-19 18:48:22 | wyoming | event=transcript data={"text": "Turn off studio light?\n[Beep]"} payload size=0
info | 2025-08-19 18:48:24 | wyoming | event=synthesize data={"text": "Sure", "voice": {"name": "autonoe"}} payload size=0
info | 2025-08-19 18:48:24 | wyoming | event=synthesize expr result=true

I have to reload the satellite

override80 avatar Aug 19 '25 19:08 override80

I've heard complaints about the stability of this new solution. Haven't worked on it yet.

AlexxIT avatar Aug 20 '25 12:08 AlexxIT

@AlexxIT I've worked around it by adding

          Sleep("1s") && WriteEvent("played") && Detect();

at the end of synthesize, just fyi

override80 avatar Aug 20 '25 12:08 override80

If you update to openwakeword 2.1 please note that by default only ok nabu keyword seems to be provided. Use this:

wake_uri: tcp://10.11.12.98:10400?name=okay_nabu

or use your custom keywork in /share/openwakeword

Hey @AlexxIT , did you change something in the last couple of weeks? It does not work anymore, no logs, no events, it's like if the binary from nightly behaves like a go2rtc version which did not introduce wyoming integration yet. I don't even get the internal-detection event.

override80 avatar Nov 11 '25 07:11 override80