[Config Support]: no audio in the stream when using 2 way audio communication with frigate lovelace card

[automjordan]

Describe the problem you are having

Hi there, i don't know if the right place to ask is here or in the repository of the lovelace frigate card but , the issue is that when i press the microphone button in the lovelace card, the audio stops and i'm not able to hear the audio from the camera.

Everything is set up according to documentation, i did the following test :

• addedd " ?video&audio " after " rtsp://admin:*****@192.168.1.129:554/h264Preview_01_sub " this allowed me to hear the sound while the mic button was active, but unfortunately disabled 2 way audio config .

I'm using :

1. go2rtc version 1.7.1 linux/amd64
2. SYSTEM 0.13.1-34FB1C2
3. VM with proxmox on nuc with coral
4. reolink doorbell wifi with last firmware v3.0.0.2555_23080700
5. config as mentioned in the documentation

Thanks for your help!

Version

SYSTEM 0.13.1-34FB1C2 Frigate (Full Access) Beta (0.13) Current version: 0.13.1

Frigate config file

mqtt:
  host: 192.168.1.10
  user: mqtt_user
  password: *******

detectors:
  coral:
    type: edgetpu
    device: usb 
rtmp:
  enabled: False
go2rtc:
  streams:
    campanello:
      - "ffmpeg:http://192.168.1.129/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=*****video=copy#audio=copy#audio=opus"
      - rtsp://admin:*****@192.168.1.129:554/h264Preview_01_sub
      #?video&audio

## THIS PREVENTS THE AUDIO COMING OUT FROM THE DOORBELL SOUNDING MUFFLED
  ffmpeg:
    bin: ffmpeg
    volume: "-af \"volume=30dB\""

##########################################################################
objects:
  filters:
    person:
      threshold: 0.70

live:
  height: 720

cameras:

##############################################
########## campanello  #######################
##############################################

  campanello:
    ffmpeg:
      inputs:
                    ## CHANGE RTSP USERNAME AND PASSWORD

        - path: rtsp://127.0.0.1:8554/campanello?video=copy&audio=aac

          roles:
            - detect
            - record
      output_args:
        record: preset-record-generic-audio-copy
    objects:
      track:
        - person
  
    detect:
      width: 1280

      height: 720
      fps: 5
      enabled: True
      max_disappeared: 25
      stationary:
        interval: 20
        threshold: 10

    snapshots:
      enabled: True
      timestamp: false
      bounding_box: true
      retain:
        default: 1

    record:
      enabled: true
#      retain:
      events:
        pre_capture: 5
        post_capture: 5
        retain:
            default: 2 
            mode: motion

Relevant log output

2024-02-04 20:19:28.959958108  [INFO] Preparing Frigate...
2024-02-04 20:19:28.969934221  [INFO] Starting Frigate...
2024-02-04 20:19:30.474904795  [2024-02-04 21:19:30] frigate.app                    INFO    : Starting Frigate (0.13.1-34fb1c2)
2024-02-04 20:19:30.475073246  [2024-02-04 21:19:30] frigate.app                    INFO    : Creating directory: /tmp/cache
2024-02-04 20:20:11.933851492  [2024-02-04 21:20:11] peewee_migrate.logs            INFO    : Starting migrations
2024-02-04 20:20:11.938097447  [2024-02-04 21:20:11] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-02-04 20:20:11.943374360  [2024-02-04 21:20:11] frigate.app                    INFO    : Recording process started: 714
2024-02-04 20:20:11.946942842  [2024-02-04 21:20:11] frigate.app                    INFO    : go2rtc process pid: 89
2024-02-04 20:20:11.977861001  [2024-02-04 21:20:11] detector.coral                 INFO    : Starting detection process: 723
2024-02-04 20:20:14.887647237  [2024-02-04 21:20:11] frigate.app                    INFO    : Output process started: 726
2024-02-04 20:20:14.887708392  [2024-02-04 21:20:11] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-02-04 20:20:14.888712131  [2024-02-04 21:20:12] frigate.app                    INFO    : Camera processor started for campanello: 753

2024-02-04 20:20:14.889335346  [2024-02-04 21:20:12] frigate.app                    INFO    : Capture process started for campanello: 825
2024-02-04 20:20:14.892809283  [2024-02-04 21:20:14] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

Frigate stats

Not found

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Any other information that may be helpful

No response

[NickM-27]

Use the recommended reolink config https://docs.frigate.video/configuration/camera_specific#reolink-cameras (including doorbell section)

[automjordan]

sorry, can you point to me the differences between my config and the official docs config?it seems both are equal to me. thanks

[NickM-27]

I can see the post was edited, video=copy#audio=copy#audio=opus was added after my comment but the first video is missing the # with that corrected it should work as expected

[automjordan]

sorry, my bad, i was doing some tests and i forgot to paste the right config i was using, then i did a second mistake forgetting the # before video. adding # does not solve the issue :

ffmpeg:http://192.168.1.129/flvport=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=#video=copy#audio=copy#audio=opus" rtsp://admin:@192.168.1.129:554/h264Preview_01_sub

[automjordan]

Any possible ideas? Thanks

[TheDescender]

I have the same doorbell and was struggling with this issue as well. I tested this directly in the go2rtc webui and frigate card and it worked for me eventually with the simple config listed in the docs.

After bashing my head against it for a couple hours I finally realized that the audio doesn't get muted when 2way is activated, but instead the device puts a fat noise gate on the microphone that removes all ambient sound. So unless you are standing at the doorbell talking into it you won't hear anything. You especially won't hear your own voice reverberate back through the mic either.

I'd strongly recommend checking the mic with 2 people. I can't post my full config at this time, but I'd recommend removing the ?video=copy&audio=copy from your camera section links. Also to each input path add: input_args: preset-rtsp-restream

Then all you have to do is make sure your frigate card has the correct reference to the desired go2rtc stream.

[automjordan]

I have the same doorbell and was struggling with this issue as well. I tested this directly in the go2rtc webui and frigate card and it worked for me eventually with the simple config listed in the docs.

After bashing my head against it for a couple hours I finally realized that the audio doesn't get muted when 2way is activated, but instead the device puts a fat noise gate on the microphone that removes all ambient sound. So unless you are standing at the doorbell talking into it you won't hear anything. You especially won't hear your own voice reverberate back through the mic either.

I'd strongly recommend checking the mic with 2 people. I can't post my full config at this time, but I'd recommend removing the ?video=copy&audio=copy from your camera section links. Also to each input path add: input_args: preset-rtsp-restream

Then all you have to do is make sure your frigate card has the correct reference to the desired go2rtc stream.

hi, let me know if i understood correctly, i removed the video=copy part : campanello: - "ffmpeg:http://192.168.1.129/flvport=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=****"

and then added in each camera configuration : input_args: preset-rtsp-restream

unfortunately i can't see any difference. I've came up with another idea, i have another camera with good mic very close to the doorbell , can i use the audio from another camera? i tried but without success. If anyone can help would be great. Thanks!

[supermarkert]

I'm using the Reolink wifi doorbell and my 2-way audio seems to work (at least from my side; haven't done extensive testing with another person to test quality). I'm using the recommended settings. A couple comments:

1. I see an error in your latest http stream URLs, as you're missing a ? between "flv" and "port". However, note that this shouldn't affect 2-way audio since only the RTSP stream is used for that.
2. You need to confirm you're always accessing your frigate/HA instance using HTTPS, since audio only works over secure connections.
3. You must be using the go2rtc and webrtc modes in the card.

My code snippets are below:

go2rtc section:

go2rtc:
  log:
    format: text
    exec: trace
  webrtc:
    candidates:
    - 192.168.40.5:8555
    - stun:8555
  streams:
    frontdoor_cam:
    - "ffmpeg:http://192.168.40.21/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_RTSP_USER}&password={FRIGATE_RTSP_PASSWORD}#video=copy#audio=copy#audio=opus"
    - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.40.21:554/h264Preview_01_sub
    frontdoor_cam_sub:
    - "ffmpeg:http://192.168.40.21/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_RTSP_USER}&password={FRIGATE_RTSP_PASSWORD}"

cameras section:

cameras:
  frontdoor_cam:
    ui:
      order: 1
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/frontdoor_cam
        input_args: preset-rtsp-restream
        roles:
        - record
      - path: rtsp://127.0.0.1:8554/frontdoor_cam_sub
        input_args: preset-rtsp-restream
        roles:
        - detect
    objects:
      track:
      - person
      - dog
      - cat
    detect:
      width: 640
      height: 480
    live:
      stream_name: frontdoor_cam_sub

my lovelace card config:

type: custom:frigate-card
cameras:
  - camera_entity: camera.frontdoor_cam
    engine: frigate
    live_provider: go2rtc
    icon: mdi:doorbell-video
    go2rtc:
      modes:
        - webrtc
live:
  auto_mute: all
  transition_effect: slide
  microphone:
    always_connected: true
  controls:
    builtin: true
    title:
      mode: none
view:
  default: snapshot
  camera_select: live
dimensions:
  aspect_ratio_mode: static
  aspect_ratio: '4:3'
menu:
  style: overlay
  buttons:
    microphone:
      enabled: true

[automjordan]

thank you very much for your help, but same behaviour is happening, voice is correctly sent to the doorbell , but the audio is immediately stopped after i press the mic button. :(

[TheDescender]

@automjordan You are 100% certain that audio is actually stopped? As in you've tested this with another person or using your phone while in front of the camera? Like I mentioned before it seems the reolink doorbell puts a huge noise gate on the mic when 2way is engaged. So you will stop hearing environmental noises.

I'd also like you to confirm that the stream from your camera is in webrtc mode. In your config you posted originally I don't see your server as one of the webrtc candidates. I recall 2way requiring the etc protocol, but I light be remembering wrong. Still doesn't hurt to configure it correctly regardless.

[automjordan]

i did try one more time going in front of the doorbell and i must say, it works! i thank you so much for the help. by the way what you mean with "In your config you posted originally I don't see your server as one of the webrtc candidates" ? i'm using frigate and i followed the docs, maybe i misunderstood something. Did you notice a strong delay (2-3 secs) with the 2 way audio communication too? thanks again

[TheDescender]

@automjordan yes there is a bit of a delay. Webrtc streaming protocol reduces the delay a fair bit as opposed to MSE and jsmpeg. In the config @supermarkert sent, you can see the candidates section under the go2rtc section. You will want to list the IP of the machine you're running frigate on as one of the candidates. That way the livestream can use webrtc for reduced delay.

[AussiSG]

Hmm seem I too have been struggling to get this working....

I do hear the outside audio when I have it enabled. But sadly enough talking through my phone doesn't go through the doorbell ....

Maybe some one can detect the flaw in my config :)

It can be found here: https://pastebin.com/v0an1cni

[supermarkert]

It can be found here: https://pastebin.com/v0an1cni

You're missing the config for the live stream. Under the config for the Doorbell camera, add this:

    live:
      stream_name: doorbell

And make sure you're accessing your instance over HTTPS for mic to work.

You can also maybe getting rid of these red parts because I don't think it's needed.

[AussiSG]

Thank you it seems that didnt do the trick. Started my own topic to not pollute this topic :) #10595 Also the problem seems different because the logs say that audio sending is active

[ffsec]

@supermarkert:

I set up everything today, but ran into a problem: When I press the microphone button on the frigate card for the first time, the blue circle on the doorbell turns on (a stream is established) and I can talk through the doorbell. If I turnthe microphone off, the stream seems to stay active and the blue circle stays on forever. At this point only the frigate card can use the speaker of doorbell, the Reolink App for example says, that it's currently being used by someone else. Only when I turn off or restart Frigate, the blue circle turns off again and the Reolink app can now be used to talk through the doorbell.

Do you have the same behavior? Which firmware are you using? I'm currently using 2676_23090700.

[AussiSG]

I've closed my topic. Apparently the latest firmware of the Reolink doorbell kills the 2 way communication of frigate.

https://github.com/blakeblackshear/frigate/discussions/10595#discussioncomment-8882296

[katschub88]

I've closed my topic. Apparently the latest firmware of the Reolink doorbell kills the 2 way communication of frigate.96)

That's good, but do you experience the same issue I've described before? Or can you use the frigate card microphone button and afterwards also use the Reolink app to speak through the doorbell? Does the blue ring on the doorbell stay on after using the frigate card microphone button?

[AussiSG]

I've closed my topic. Apparently the latest firmware of the Reolink doorbell kills the 2 way communication of frigate.96)

That's good, but do you experience the same issue I've described before? Or can you use the frigate card microphone button and afterwards also use the Reolink app to speak through the doorbell? Does the blue ring on the doorbell stay on after using the frigate card microphone button?

Indeed I can confirm this one ! It seems the 2 way communication stays established and the reolink can't take over

edit: Although after a while I can take the doorbell over with reolink. But indeed the connections remains on for a while even after disabling mic/audio in the frigate card

[AussiSG]

@NickM-27 the above mentioned error by @katschub88 also occurs when I'm opening the frigate card of the doorbell. I do this when the doorbell is pressed. After this even the doorbell can't be pressed anymore for action because it's being used by frigate.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.