AlexxIT
Choppy Audio in HomeKit
Hey there,
I'm trying to add a camera stream to HomeKit but am noticing that the audio cuts out every ~3 seconds. AFAICT, this issue is specific to HomeKit as the stream's audio works correctly in VLC (using the raw RTMP stream) and Frigate (using the go2rtc re-stream). Here's the config I'm using:
go2rtc:
streams:
nanit:
- rtmp://<cameraIP>
homekit:
nanit:
name: Nanit
pin: <pin>
Here's the info on the RTMP stream from VLC:
I've tried re-encoding the stream using ffmpeg:rtmp://<cameraIP>#video=copy#audio=opus and ffmpeg:rtmp://<cameraIP>#video=copy#audio=aac, but neither seems to address the issue. I've also tried adding /16000 to the end of the audio parameter to force a specific sample rate.
My understanding is that this should have been fixed in #667, but is there something obvious I'm missing here? Or logs I can provide to help debug? Thanks in advance!
Try to use #audio=opus/16000
https://github.com/AlexxIT/go2rtc/blob/753d6617ab582cc8b59f0f27968181440cd9d735/pkg/hap/camera/ch115_supported_audio.go#L23-L25
Try to use
#audio=opus/16000https://github.com/AlexxIT/go2rtc/blob/753d6617ab582cc8b59f0f27968181440cd9d735/pkg/hap/camera/ch115_supported_audio.go#L23-L25
I tried that, but still no luck:
ffmpeg:rtmp://<cameraIP>#video=copy#audio=opus/16000
One thing I just noticed in the go2rtc WebUI is that adding #audio=opus/16000 doesn't seem to actually change the audio codec sample rate. Is this expected?
EDIT: Nvm. Just realized that per the Opus spec, audio will always be at 48 kHz and this parameter just impacts band limiting. Is there something else I should be trying?
{
"producers": [
{
"id": 62,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:56930 forwarded 192.168.1.101:1935",
"source": "exec:/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -i rtmp://192.168.1.101:1935 -c:v copy -c:a libopus -application:a lowdelay -min_comp 0 -ar:a 16000 -ac:a 1 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/5a8cb3ba73ff45c787a0430b40268272",
"url": "rtmp://192.168.1.101:1935",
"sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=No Name\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat 61.1.100\r\nm=video 0 RTP/AVP 96\r\nb=AS:3145\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z01AKI2NQDwBE/LgLcBAQFAAAD6AAATiDoYA/8AAF9eC7y40MAf+AAC+vBd5cKA=,aO44gAA=; profile-level-id=4D4028\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\nb=AS:64\r\na=rtpmap:97 opus/48000/2\r\na=control:streamid=1\r\n",
"user_agent": "ffmpeg/go2rtc",
"medias": [
"video, recvonly, H264",
"audio, recvonly, OPUS/48000/2"
],
"receivers": [
{
"id": 63,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 40,
"profile": "Main"
},
"childs": [
65,
67,
116
],
"bytes": 26077979,
"packets": 18975
},
{
"id": 64,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"childs": [
66,
68,
117
],
"bytes": 1550723,
"packets": 10419
}
],
"bytes_recv": 27983726
}
],
"consumers": [
{
"id": 13,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:56742",
"sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.10\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z01AKI2NQDwBE/LgLcBAQFAAAD6AAATiDoYA/8AAF9eC7y40MAf+AAC+vBd5cKA=,aO44gAA=; profile-level-id=4D4028\r\na=recvonly\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 OPUS/48000/2\r\na=recvonly\r\na=control:trackID=1\r\n",
"user_agent": "FFmpeg Frigate/0.17.0-430cebe",
"medias": [
"video, sendonly, ANY",
"audio, sendonly, ANY"
],
"senders": [
{
"id": 65,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 40,
"profile": "Main"
},
"parent": 63,
"bytes": 26077979,
"packets": 18975
},
{
"id": 66,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"parent": 64,
"bytes": 1550723,
"packets": 10419
}
],
"bytes_send": 28099006
},
{
"id": 27,
"format_name": "rtsp",
"protocol": "rtsp+tcp",
"remote_addr": "127.0.0.1:56888",
"sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.10\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z01AKI2NQDwBE/LgLcBAQFAAAD6AAATiDoYA/8AAF9eC7y40MAf+AAC+vBd5cKA=,aO44gAA=; profile-level-id=4D4028\r\na=recvonly\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 OPUS/48000/2\r\na=recvonly\r\na=control:trackID=1\r\n",
"user_agent": "FFmpeg Frigate/0.17.0-430cebe",
"medias": [
"video, sendonly, ANY",
"audio, sendonly, ANY"
],
"senders": [
{
"id": 67,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 40,
"profile": "Main"
},
"parent": 63,
"bytes": 26077979,
"packets": 18975
},
{
"id": 68,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"parent": 64,
"bytes": 1550723,
"packets": 10419
}
],
"bytes_send": 28099006
},
{
"id": 115,
"format_name": "probe",
"protocol": "http",
"remote_addr": "192.168.1.29:54172",
"user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.2 Mobile/15E148 Safari/604.1",
"medias": [
"video, sendonly, ALL",
"audio, sendonly, ALL",
"audio, recvonly, ANY"
],
"senders": [
{
"id": 116,
"codec": {
"codec_name": "h264",
"codec_type": "video",
"level": 40,
"profile": "Main"
},
"parent": 63
},
{
"id": 117,
"codec": {
"channels": 2,
"codec_name": "opus",
"codec_type": "audio",
"sample_rate": 48000
},
"parent": 64
}
]
}
]
}