blakeblackshear
[Support]: Transcoded audostream issue go2rtc
Describe the problem you are having
I've setup the new go2rtc config within my config.yml and specified the option to provide an opus audiostream in order to to test it out with webrtc however the audio is very choppy/broken up. The audio sounds exactly the same as the previous implementation when there was only aac preesnt direct from the camera (which works fine in mse)
go2rtc:
streams:
Drive: ffmpeg:rtsp://admin:*****@10.0.0.110:554/Streaming/Channels/101/#video=copy#audio=copy#audio=opus
I also tried removing the original stream and only providing the opus version#video=copy#audio=opus which results in the same choppy/corrupted audio in webrtc and no audio in mse as I would expect.
nothing showing in the logs. See details of the audio stream coming from the camera in the ffprobe output.
Version
0.12.0-5834233
Frigate config file
#frigate sample config.yml
mqtt:
# Required: host name
host: 10.0.0.11
# Optional: port (default: shown below)
port: 1883
# Optional: topic prefix (default: shown below)
# WARNING: must be unique if you are running multiple instances
topic_prefix: frigate
# Optional: client id (default: shown below)
# WARNING: must be unique if you are running multiple instances
client_id: frigate
# Optional: user
user: assistant
# Optional:
# NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
# eg. password: '{FRIGATE_MQTT_PASSWORD}'
password: 1nt3rc00l3r
# Optional: interval in seconds for publishing stats (default: shown below)
stats_interval: 60
birdseye:
# Optional: Enable birdseye view (default: shown below)
enabled: False
model:
labelmap:
16: cat
17: cat
#ui:
# use_experimental: true
go2rtc:
streams:
Drive: ffmpeg:rtsp://admin:*@10.0.0.110:554/Streaming/Channels/101/#video=copy#audio=copy#audio=opus
Drive_Left: rtsp://admin:*@10.0.0.100:554/Streaming/Channels/101/
Garden: rtsp://admin:*@10.0.0.241:554/Streaming/Channels/101/
webrtc:
candidates:
- 10.0.0.10:8555
- stun:8555
cameras:
# Doorbell:
# ffmpeg:
# hwaccel_args:
# - -hwaccel
# - qsv
# - -qsv_device
# - /dev/dri/renderD128
# - -c:v
# - h264_qsv
# input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport udp -timeout 3500000 -use_wallclock_as_timestamps 1
# inputs:
# - path: rtsp://10.0.0.12:554/doorbell
# roles:
# - restream
# - record
# - detect
Drive:
mqtt:
timestamp: False
bounding_box: False
crop: True
quality: 100
height: 600
ffmpeg:
hwaccel_args:
- -hwaccel
- qsv
- -qsv_device
- /dev/dri/renderD128
- -c:v
- h264_qsv
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
output_args:
detect: -vf hwdownload,format=nv12 -f rawvideo -pix_fmt yuvj420p
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
inputs:
- path: rtsp://admin:*@10.0.0.110:554/Streaming/Channels/101/
roles:
- record
- path: rtsp://admin:*@10.0.0.110:554/Streaming/Channels/103/
roles:
- detect
objects:
track:
- person
- car
- cat
filters:
person:
min_area: 1000
min_score: 0.8
car:
min_score: 0.2
threshold: 0.5
mask:
# - 2926,621,3016,867,3840,2160,3840,0,0,0,0,2160,3840,2160,2991,839,2012,649,2311,490,2308,265,2944,343
motion:
mask:
#4k - 2095,173,2938,323,3071,378,3840,858,3840,0,1701,0,0,0,0,441,1114,339,1118,217
- 0,0,0,1080,189,237,527,165,567,156,551,103,646,93,825,103,982,90,1181,102,1185,137,1484,174,1714,188,1920,1080,1920,0
snapshots:
required_zones:
- entrance
- everywhere
enabled: true
timestamp: true
bounding_box: true
crop: true
quality: 100
height: 600
retain:
default: 5
detect:
width: 1920
height: 1080
record:
enabled: True
events:
required_zones:
- entrance
- everywhere
retain:
default: 5
objects:
cat: 2
zones:
entrance:
#4k coordinates: 1992,642,2997,835,2941,602,2953,284,2355,209,2323,437
coordinates: 1485,333,1134,254,1165,170,1477,223
objects:
- car
everywhere:
#4k coordinates: 2350,217,2365,461,2942,631,2949,0,3840,543,3840,2160,0,2160,0,0,1040,0,1055,331,1158,299,1142,173
coordinates: 1920,0,1920,1080,0,1080,0,0,517,188,550,227,620,203,685,184,866,164,992,177,1108,194,1103,257,1519,359,1630,0
objects:
- person
- cat
Drive_Left:
mqtt:
timestamp: False
bounding_box: False
crop: True
quality: 100
height: 600
ffmpeg:
hwaccel_args:
- -hwaccel
- qsv
- -qsv_device
- /dev/dri/renderD128
- -c:v
- h264_qsv
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
output_args:
detect: -vf hwdownload,format=nv12 -f rawvideo -pix_fmt yuvj420p
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
inputs:
- path: rtsp://admin:*@10.0.0.100:554/Streaming/Channels/101/
roles:
- record
- path: rtsp://admin:*@10.0.0.100:554/Streaming/Channels/103/
roles:
- detect
objects:
track:
- person
- cat
- car
filters:
cat:
min_score: 0.3
threshold: 0.55
snapshots:
required_zones:
- left_entrance
- left_everywhere
enabled: true
timestamp: true
bounding_box: true
crop: true
quality: 100
height: 600
retain:
default: 5
detect:
width: 1280
height: 720
record:
enabled: True
events:
required_zones:
- left_entrance
- left_everywhere
retain:
default: 5
objects:
cat: 2
motion:
mask:
- 731,23,538,36,87,270,0,0,1077,0,1057,79
zones:
left_entrance:
coordinates: 1072,237,843,189,955,62,1086,81
objects:
- car
left_everywhere:
coordinates: 635,34,982,79,920,194,1074,228,1121,0,1280,0,1280,720,0,720,0,296,290,191
objects:
- person
- cat
Garden:
mqtt:
timestamp: False
bounding_box: False
crop: True
quality: 100
height: 600
ffmpeg:
hwaccel_args:
- -hwaccel
- qsv
- -qsv_device
- /dev/dri/renderD128
- -c:v
- h264_qsv
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
output_args:
detect: -vf hwdownload,format=nv12 -f rawvideo -pix_fmt yuvj420p
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
inputs:
- path: rtsp://admin:*@10.0.0.241:554/Streaming/Channels/101/
roles:
- record
- path: rtsp://admin:*@10.0.0.241:554/Streaming/Channels/103/
roles:
- detect
motion:
threshold: 8
mask:
#4k - 0,0,0,851,571,579,953,480,1382,378,1756,370,2339,343,3052,358,3103,311,3221,276,3410,260,3410,146,3575,0
- 0,0,0,437,116,398,315,309,386,290,591,235,866,185,1147,178,1548,184,1608,138,1722,140,1920,0
objects:
track:
- person
- cat
filters:
cat:
min_score: 0.3
threshold: 0.55
person:
min_score: 0.8
threshold: 0.75
min_area: 1000
mask:
- 412,335,449,486,495,466,461,319
zones:
all_garden:
#4k coordinates: 1992,642,2997,835,2941,602,2953,284,2355,209,2323,437
coordinates: 1920,0,0,0,0,1080,1920,1080,1920,322,1920,0,1723,520,1629,505,1640,363,1725,305,1737,492
objects:
- cat
- person
doorstep:
#4k coordinates: 3840,1465,3398,1355,3489,819,3840,851
coordinates: 1729,317,1634,331,1623,517,1728,526
objects:
- cat
- person
snapshots:
quality: 100
height: 600
enabled: true
timestamp: true
bounding_box: true
crop: true
retain:
default: 5
detect:
width: 1920
height: 1080
record:
enabled: True
events:
retain:
default: 5
objects:
cat: 2
detectors:
coral_pci:
type: edgetpu
device: pci
# cpu1:
# type: cpu
Relevant log output
N/A
FFprobe output from your camera
Stream #0:1: Audio: aac (LC), 44100 Hz, mono, fltp
Frigate stats
No response
Operating system
UNRAID
Install method
Docker CLI
Coral version
PCIe
Network connection
Wired
Camera make and model
hikvision
Any other information that may be helpful
No response
I've not had any issues with opus, but there's not much we can do to change how that is working. You can also try #audio=pcmu and see how that works out.
Strangely it's the exact level of choppiness/stuttering just lower fidelity as expected with pcmu. If I drop the AAC sampling rate to 16khz and the bitrate down to 16kbps on the camera it reduces the issue but it's still noticeably stuttery unfortunately and in doing so I'm losing the higher audio quality.
Interesting, maybe make an issue on go2rtc but most likely just be the case that ffmpeg isn't transcoding it well
Yeah, I'm leaning more toward it being ffmpeg struggling to transcode it decently. I'll open an issue regardless.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}