frigate-hass-card
frigate-hass-card copied to clipboard
dermotduffy
2-way audio does not work with certain Hikvision cameras
Checklist:
- [X] I updated to the latest version available
- [X] I cleared the cache of my browser
Release with the issue: Specific PR release: https://github.com/dermotduffy/frigate-hass-card/pull/1291 But it also happens with 5.2.0 and Frigate addon
Last working release (if known): Unknown Browser and Operating System: This happens on all browsers mac/chrome/companion app
Description of problem:
Using the config below, two way audio works, toggling the MIC button on/off again, always stops/starts the two way audio... But the camera is also a doorbell, when a call is received, we need to first mute the call, by sending a service to it, to make the ringing sound stop, afterwards we press the mic button to start talking... so far so good.. Then i press the mic button to stop the audio, this also works... Next on the second call, we do the same thing, we send the service to the doorbell to stop the ringing, but the when we press the MIC button, the two way audio isnt initiated... this is my issue What we need todo, is to refresh the page where the frigate card is located, after the refresh, when we then press again the MIC button, it seems to work....
I dont know why, or whats the cause, or why i need to refresh the page?
frigate card config:
- type: custom:frigate-card
cameras:
- camera_entity: camera.deurbel
live_provider: go2rtc
go2rtc:
url: https://user:[email protected]:8123
stream: deurbel
modes:
- webrtc
......
go2rtc config:
streams:
deurbel:
- rtsp://user:[email protected]:554/Sms=24.unicast
- isapi://user:[email protected]:80/
I also checked the go2rtc addon logs, no isses there... so not sure if its go2rtc or frigate card related, difficult to tell.. What is happening on a refresh of the view? whats different between a refresh of the view and toggling the mic button?
What i also see, when you do a refresh of the page, the mic button is not immediately visible, it takes a second, and then its there... Is there maybe a way to force that refresh? That could maybe solve my issue, alltough still dont know the root cause...
Edit:
My guess, go2rtc is using ISAPI to use two way audio on hikvision camera's, I think the audio session is not closed when pressing the mute button? When refreshing the page it starts a new two way audio session? The audio session is always open, regardless the mic button is muted or not?? That way, on an incoming call the audio session is closed, bur frigate thinks it's still open?
Just to add to the above (I've been working with @pergolafabio, I have a Hikvision Doorbell and Internal Intercom. I am using the Frigate add-on, Frigate Integration and Hass card to use the camera, speaker and microphone from the Hikvision Doorbell. When using this all through the Frigate Hass card, it all initially works well and I get 2 way audio.
However, when I take the following steps things go wrong:
- Checked that 2 way audio working
- Press doorbell button
- Pressed the HA entity indoor button for answer call (Hikvision Doorbell Entity)
- Pressed the HA entity indoor button for hangup call (Hikvision Doorbell Entity)
- 2 way audio not working anymore
- Restarted Frigate / reload Hass card
- 2 way audio back again
I'm happy to do any testing.
a quick update, i tested the WebRTC card to rule out go2rtc, and that card works reliable everytime i click the mute/unmute icon to switch streams, it works
is it maybe possible to compare the mic button from frigate to behave also like the webrtc card?
My guess, go2rtc is using ISAPI to use two way audio on hikvision camera's, I think the audio session is not closed when pressing the mute button?
Correct.
When refreshing the page it starts a new two way audio session? The audio session is always open, regardless the mic button is muted or not??
Unless you use the always_connected option, the stream will be converted to a 2-way session the first time the microphone is unmuted. It will automatically disconnect (convert back to without 2-way audio) after disconnect_seconds have elapsed.
Next on the second call, we do the same thing, we send the service to the doorbell to stop the ringing, but the when we press the MIC button, the two way audio isnt initiated... this is my issue
So it all works fine, until to call an unrelated service and then the microphone unmute doesn't work anymore? How are you calling the unrelated service? Can you provide your full card config?
Thank you!
Hey , thnx for feedback already, appreciate it! Below full config As you can see, im using the dev card version, where i can point to camera without frigate addon/integration
As you can see, i have extra 2 buttons ( see also screenshot) a phone / hangup button That phone buttons calls a script, so that when someone is actually pressing the doorbell button, i need to stop the ringing and afterwards inittiate the two way audio
But i also see this new PF : https://github.com/dermotduffy/frigate-hass-card/pull/1359 That means i can combine the answer + mic button into one, thnx already for that, will test that out later!
But anyway, back to issue, you are talking about always_connected , i tried it with true/false, but ddint make any difference... But i wasnt aware of the disconnect_seconds!! I was always testing making calls after each other, within the 60 seconds
What is the best approach for this? probably setting this back to "false" , so that means, on incoming call, if i wait longer then 60 seconds for next call , it should work? What happens if the stream is active? I mean if the lovelave card is active? Does it trigger the 60 sec disconnected? Or does it only disconnect if i browse away from the page?
That being said, maybe that new PR could be helpfull with the automation? Can i define for example, if press the hangup button or if i toggle again the mic button, that i force the disconnect?
But whats still strange, i dont know if the disonnect_seconds will solve it... When the issue occured, i could toggle the mic button as many times if i want, it never worked again sending two way audio, untill i did a refresh of the lovelace card... Shouldnt toggling the mic button , disconnect any open audio sessions anway?
- type: custom:frigate-card
cameras:
- camera_entity: camera.deurbel
live_provider: go2rtc
go2rtc:
url: https://user:[email protected]:8123
stream: deurbel
menu:
style: outside
position: bottom
buttons:
microphone:
enabled: true
type: toggle
screenshot:
enabled: false
download:
enabled: false
fullscreen:
enabled: false
snapshots:
enabled: false
timeline:
enabled: false
media_player:
enabled: false
clips:
enabled: false
live:
enabled: false
cameras:
enabled: false
frigate:
enabled: false
camera_ui:
enabled: false
live:
microphone:
always_connected: true
auto_mute: []
auto_unmute: []
controls:
builtin: true
title:
mode: none
layout:
fit: fill
elements:
- type: custom:frigate-card-menu-icon
icon: mdi:volume-high
tap_action:
- action: custom:frigate-card-action
frigate_card_action: unmute
- type: custom:frigate-card-menu-icon
icon: mdi:volume-off
tap_action:
- action: custom:frigate-card-action
frigate_card_action: mute
- type: custom:frigate-card-menu-icon
icon: mdi:phone
tap_action:
- action: call-service
service: script.answer_call
- action: custom:frigate-card-action
frigate_card_action: unmute
- action: custom:frigate-card-action
frigate_card_action: microphone_unmute
- type: custom:frigate-card-menu-icon
icon: mdi:phone-hangup
tap_action:
- action: custom:frigate-card-action
frigate_card_action: microphone_mute
- type: custom:frigate-card-menu-icon
icon: mdi:door-open
hold_action:
- action: call-service
service: switch.turn_on
data:
entity_id: switch.ds_kd8003_door_relay_0
dimensions:
aspect_ratio_mode: static
aspect_ratio: '16:9'
For quick testing, i changed my card to this setting:
live:
microphone:
always_connected: false
disconnect_seconds: 15
Indeed after 15 sec, the mic is disconnected, i can turn on mic and speak again, so that setting is working
However , my issue is still present, if i have a real call coming in, i drop the doorbell sound, but then i enable mic, but i cant speak... if i then wait 15 sec, the mic is off, i turn it on again, i still cant speak
The only solution, is to browse away from the lovelace card or a a refresh of the page, then i can speak again
There must be something different in refreshing the page and the disconnect feature, i think the dsconnect feature doesnt disconnect enough? Whats the difference between your disconnect feature and refreshing the page?
@pergolafabio maybe you should try again with https://github.com/dermotduffy/frigate-hass-card/pull/1362.
Tried the new dev , but still doesnt work as expected for me :-) Whenever the doorbell was ringing, i cant use two way audio anymore Need to close companion app or do a hard refresh on the card to make it work again
hey @dermotduffy , any thing i can test or do ? thnx
Sorry, as I don't these cameras, I have no way to test and I'm not sure what is going on or what else to suggest I'm afraid. There are a few other 2-way audio related issues in the backlog that I will look into that could impact this, so I'll leave this issue open in case they help / in case someone else manages to make some progress to figure out what's going on.
Ok, no problem Maybe it's worth a look how the webrtc card works? Cause when I test that card pressing the mic/mute button every time works on that card.. Maybe they use a different approach to stop/start audio?
Maybe it's worth a look how the webrtc card works? Cause when I test that card pressing the mic/mute button every time works on that card..
Sounds like a great plan. Thanks.
Thnx!
If you want, I can give you access to my intercom, so you can test it remote
is it maybe related to this issue?
https://github.com/dermotduffy/frigate-hass-card/issues/1389
maybe the connection is indeed still open, and only works again after a new hard refresh of the page?
Maybe this will fix this issue: https://github.com/AlexxIT/go2rtc/pull/967
Aha, that's interesting!! I keep an eye on it, that could be it indeed
@felipecrs , thnx for pointing that issue, it fixes the issue, seems indeed related to go2rtc , and indeed a fix is underway can close this one now...
thxn for all help
I would suggest you mention you tried the fix in go2rtc repo, so that the owner can be more confident to merge it.
hmm, he didnt create an addon repo, not sure how i can test a specifc PR , since its not merged yet in alex master
Hey @dermotduffy , i did some digging and traced it with wireshark The difference with the WEBRTC card, toggling the mic button, initiates a new stream With your card, toggling the MIC button just mutes/unmutes the stream
Is there a way/service to restart the stream with your card? That could solve the issue
thnx
Yes and no, toggling the mic button on frigate, doesn't start a new stream session, I wiresharked this
Webrtc card does it, not frigate card
Hey @dermotduffy
You said: FYI: The Frigate card does restart the stream on the first press of the unmute button
Well, that's exactly the issue, that's why I only hear two way audio the first time I use the card :-) Is there some service to force to restart the stream again when toggling the mic button again?
This is very specific for Hikvision, in normal circumstances , it's not an issue , I can toggle the mic button as many times I want, I always have two way audio.. but this device is also an intercom... So when someone is making an intercom event by ringing the device, the two way audio is initiated to another client... That's why the audio is closed on the frigate card.. in order to get audio back, I need to restart the stream, that will send an isapi command (close audio) to the camera, that command is now included in the go2rtc addon since 1.9.0
Just to chime-in as a non-doorbell Hikvision camera user: It is important to me that functionality not change when "always-connected" is enabled. More specifically that the mute button does not close/restart the stream. We use this camera as door intercom and often have some back-and forth with whoever is there. Muting/Unmuting multiple times is typical.
Using AlexxxIT's WebRTC card restarts the stream on every mute/unmute. The Muting/Unmuting process is slow on the camera-end and this quickly breaks things. The ISAPI interface cannot keep up.
What I would propose is an alternate button for enable/disable mic (via stopping/starting ISAPI stream) as opposed to mute/unmute which simply mutes the client-side mic, which is instantaneous and expected behavior for any audio client/server setup.
As additional thought: Being "always-connected" should really already be the default behavior. If there is a microphone configured, the expectation is that use of that microphone should be instantaneous. Tearing down/re-building the stream is going to cause a noticeable lag. This is particularly evident when using this remotely on a mobile device...it just doesn't work.
As long as there is an "always-connected" option, the solution will work for our use case, but it took a couple month's worth of headaches before I was lucky enough that @dermotduffy gave me a heads up in another thread. Anyone newly trying this out and lacking the skills to do tcpdumps and review them via wireshark will likely find the solution unworkable.
Food for thought.
As additional thought: Being "always-connected" should really already be the default behavior. If there is a microphone configured, the expectation is that use of that microphone should be instantaneous.
This triggers the browser to think the microphone is in use, and on Android this means also highlighting the status bar item.
For example, I only use microphone 0.01% of the times I open my dashboard, so IMO it should not be made the default.
Instead, maybe investing time into somehow making attaching the microphone to an existing stream session on the fly would be a better replacement for always_connected, without requiring a stream restart.
maybe investing time into somehow making attaching the microphone to an existing stream session
@felipecrs: This is what @dermotduffy was discussing earlier. This is not possible. Nor is it correct from a development perspective. If the microphone is set to "enabled" in the config, it should be enabled. In this context that explicitly means "always-connected". Otherwise the "mute" button should be changed to "Disabled/Enabled". It is no longer a mute button.
If you personally only use the microphone rarely, then that by definition is an edge case and should carry the edge-case configuration requirement vs the expectation that if microphone is set to "enabled" that it is enabled and will remain enabled until explicitly set as "disabled", and the mute button actually means "mute" the same as how a mute button works on a phone, stereo, or any other audio device.
On your stereo, when you hit mute, the music keeps playing...but it just stops coming out of the speakers. When you hit mute on your phone, it does not disconnect the call...it simply prevents audio from being transmitted. The configuration you are recommending is that every time someone hits "mute" the phone hangs up and a new phone call needs to be made when the user wants to speak again. It might happen much faster here...especially if you are on the same network...but tearing down and rebuilding the connection on every press of the mute button just doesn't make sense.
With the current setup the expected behavior is obfuscated. The microphone is sluggish, like a child's walkie-talkie, and the connection becomes unusable regularly without any explanation...because the mute button is causing the connection to be torn-down/re-built on every press. Whereas if "always-connected" was the default, then you might have a microphone enabled on your desktop but you at least have an indicator of what is going on and can then look for the setting to change...and can then enable/disable the microphone entirely as you wish.
There is a value to have a new/special button to enable/disable the microphone entirely...but it should not be the mute button.
I should also note that tear-down/re-connection also means the video channel. They are coupled.
In no context does anyone expect that hitting "mute" means restarting the video.