Configure 2-way-audio within Home Assistant

[dm82m]

I am using a Tapo C320 cam. My setup is:

1. I installed the HomeAssistant-Tapo-Control (https://github.com/JurajNyiri/HomeAssistant-Tapo-Control) that provides the SD and HD stream of my Tapo cam within Home Assistant. My Tapo cam is not connected to the cloud, so I use the local user/password.
2. I installed WebRTC with embedded go2rtc and using the custom:webrtc-camera to get a legless video stream that is provided through the above point.

So far so good! I get a ring signal and can view the video stream in my Home Assistant. If I want to talk to the person in front of my door, I need to switch to the Tapo app to do so.

Now I have read that go2rtc supports two way audio but to be honest: I don't get it running. What I did so far:

1. I created a tapo stream within go2rtc.yaml named tapo.
2. I added a virtual media_player based on that tapo stream into configuration.yaml.
3. I have no idea how I can play audio to the cam or even enable 2-way-audio to speak with the person in front of the cam.

And now I dont know how I can put all together or I even dont know if my usecase will be possible: Instead of only showing the legless stream in Home Assistant I am directly able to use 2-way-audio to speak into the camera.

Thanks for any help!

[HAuser1234]

Hello I am also trying to enable this. actually I somehow made it work (at least to send tts messages) once but after adding a second camera it didnt work. Even if I removed it. maybe @AlexxIT can look into it. It can't be much because it did work for me once. Somewhere I somehow removed or changed something that breaks it.

This error happens when I try to play something:

Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/webrtc/media_player.py:74
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 12:25:20 (24 occurrences)
Last logged: 12:25:32

[140327525726192]
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 200, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1787, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1824, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 680, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 968, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 720, in _handle_entity_call
    await result
  File "/config/custom_components/webrtc/media_player.py", line 74, in async_play_media
    assert r.ok
AssertionError

The go2rtc.yaml looks like:

streams:
  cameraeins: tapo://[email protected]

and the configuration.yaml in HA:

media_player:
  - platform: webrtc
    name: cameraeins
    stream: cameraeins
    audio: PCMA/8000

[HAuser1234]

I have also tried opening the API in the browser (not sure if this could work) butt the result is null: http://192.168.XX.XX:1984/api/streams?dst=cameraeins&src=ffmpeg:homeserver2:8123/local/Test2.wav#audio=PCMA/8000#input=file

[AlexxIT]

This request can't be send from browser. It should be post with proper handling character #

[HAuser1234]

ok I just tried it if it could help somehow. Can I help or am I able to help to test where the problem is? Also if I send the request via postman as a post nothing happens

[dm82m]

@AlexxIT can you give any hints if my usecase would work out and if so, how? See my first post in this thread.

[HAuser1234]

@dm82m according to the docs this should be possible, although ist seems to be required to have https access to the HA server. My endgoal is exactly the same as yours, but I thought before I try to go further the basics should work, but they don't in my case. Have you got your camera to play a audio file?

[dm82m]

My instance is accessible via https, yes. No I am not able to play an audio file to the cam via media_player.

[HAuser1234]

I think I found the problem. @AlexxIT do you know why there is the error Unable`` to find a suitable output format for 'PCMA/8000'

2023-02-27 11:51:10.781 DEBUG (webrtc) [custom_components.webrtc.utils] [90m11:51:10.780[0m [33mDBG[0m [exec] run [36murl=[0m"exec:ffmpeg -hide_banner -v error -re -i http://192.168.XX.XC:8123/api/tts_proxy/b922871f61bd7f7d391db8b13d97ece04bd90198_de-de_-_picotts.wav -vn PCMA/8000 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/db87d776682d35a1d524894ae1c8104a" 2023-02-27 11:51:10.811 DEBUG (webrtc) [custom_components.webrtc.utils] [NULL @ 0x7fa5f3c84500] Unable to find a suitable output format for 'PCMA/8000' 2023-02-27 11:51:10.811 DEBUG (webrtc) [custom_components.webrtc.utils] PCMA/8000: Invalid argument

the error occurs because the Tapo camera needs theaudiostandard pcma not as in the docs PCMA\8000 It works now!!

[AlexxIT]

Oh. Your problem is PCMA/8000. There is no such template. There are pcma, pcma/48000 and some others. PCMA/8000 is the codec. pcma is the template for this codec. Check my examples carefully.

[HAuser1234]

I took the info out of here. Ok I understand that now. Maybe this should be edited or explained there because it is not really obvious to beginners. Just as a proposal

by the way is there a possibility to increase the volume. the siren is much louder, so maybe the speaker is able to be turned up(?)

[dm82m]

What kind of file can you now play on the tapo? But still no mic input from Ha? Right?

[AlexxIT]

@HAuser1234 There's a line above (which is not in your screenshot): "you should use ffmpeg source for transcoding...". And link to ffmpeg source docs, including audio templates.

You can control volume from Tapo app. Or check Tapo integration. I don't know if it has this function.

[AlexxIT]

@dm82m https://github.com/AlexxIT/go2rtc#stream-to-camera

"you can stream: local files, web files, live streams or any format, supported by FFmpeg"

[HAuser1234]

@HAuser1234 There's a line above (which is not in your screenshot): "you should use ffmpeg source for transcoding...". And link to ffmpeg source docs, including audio templates.

You can control volume from Tapo app. Or check Tapo integration. I don't know if it has this function.

ah ok that was just not obvious to me sorry :)

the audio is at max loudess. So I guess thats the limit, thanks!!

[dm82m]

works for me aswell. only thing that I know need is a home assistant card that shows the camera video and has two-way-audio support. but I think it is not existing yet.

[yarosman]

works for me aswell. only thing that I know need is a home assistant card that shows the camera video and has two-way-audio support. but I think it is not existing yet.

Hello, can you share you final configurations ?

[dm82m]

go2rtc.yaml:

streams:
  tapo: tapo://[email protected]

configuration.yaml:

media_player:
  - platform: webrtc
    name: tapo_virtual
    stream: tapo
    audio: pcma 

With that you have a new virtual media player based on the go2rtc stream and are able to cast audio to it. Please keep in mind that there is still no home assistant card that allows you to use bi-direct audio directly.

[AlexxIT]

There is card with two way audio https://github.com/dermotduffy/frigate-hass-card

[dm82m]

There is card with two way audio https://github.com/dermotduffy/frigate-hass-card

Any idea how the exact configuration flow look like?

[eded333]

There is card with two way audio https://github.com/dermotduffy/frigate-hass-card

Any idea how the exact configuration flow look like?

You need to setup frigate, then in frigates config add this: go2rtc: streams: YourCameraNameInFrigate: - rtsp://USER:[email protected]:554/stream1 or 2 - tapo://admin:MD5 Hash of your cloud password [email protected]

Then go to your frigate card, add your frigate camera, force it to go through go2rtc and make it only available with webrtc format, then enable mic in the card options and browser and access it with https.

You could do the same with the go2rtc addon, using the same go2rtc config, this guy explains it enough: https://www.youtube.com/watch?v=WnRJxneCUYE

[hoanglongutc]

There is card with two way audio https://github.com/dermotduffy/frigate-hass-card

I used fridgate, but still can't talk two way (even though the microphone icon is activated) I tested with "https:///api/hassio_ingress/5aPWLsxgC_oYCPNaHiu4rGBNgatzgGUkRRLYPU/webrtc.html?src=hikvision_pk&media=video+audio+microphone" and "https://alexxit.github.io/go2rtc/#share=Kd2eBgh&pwd=hnwK**&media=video+audio+microphone" , still two-way conversation is normal

[tsrubar]

same here. mic icon is on, but no sound output from doorbell edit: ok, i found problem in browser, when chrome didnt allow to access mic on http site. after following this guide, 2way started to work

[TomK]

possibly relates to https://github.com/dermotduffy/frigate-hass-card/issues/1235

[calisro]

the audio is at max loudess. So I guess thats the limit, thanks!!

You could increase the volume with ffmpeg if I read what you said correctly... If not, ignore. My reolink needed padding to work 'okay' so you can include or ignore that part.

  - platform: webrtc
    name: Frontdoor
    stream: frontdoor_send
    audio: '-af "volume=10dB,adelay=2s,apad=pad_dur=6" -c:a pcm_alaw -ar:a 8000 -ac:a 1'

or within the go2rtc.yaml:

ffmpeg:
  opus: '-af "volume=10dB" -c:a libopus -b:a 16K -vbr constrained -application:a lowdelay -frame_duration 20 -min_comp 0'