homebridge-videodoorbell
homebridge-videodoorbell copied to clipboard
Samfox2
Two way audio ?
Is it possible? No one camera makers even "Works with homekit" logo dont have two way audio feature. Somfy is announce, but sales is not starting yet. Is it homekit problem or not? Seems that happened when apple removes doorbell category from list...
I have my video doorbell currently working.
https://github.com/KhaosT/homebridge-camera-ffmpeg/issues/174
Production environment is:
Raspberry PI2 B
Linux TeoRasp 4.14.70-v7+ #1144 SMP Tue Sep 18 17:34:46 BST 2018 armv7l
node v8.11.1
npm -g -depth=0 list /usr/lib ├── [email protected] ├── [email protected] ├── [email protected] ├── [email protected] ├── [email protected] └── [email protected]
libasound2-dev
ffmpeg version N-91008-ge6114d21ac Copyright (c) 2000-2018 the FFmpeg developers built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516 configuration: --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree --enable-mmal --enable-libfdk-aac
{
"platform": "Video-doorbell",
"cameras": [
{
"name": "videodorbell test",
"port": 5005,
"button": true,
"videoConfig": {
"source": "-f alsa -i hw:CARD=usbtv -f v4l2 -i /dev/video0",
"stillImageSource": "-nostats -nostdin -timelimit 2 -analyzeduration 1 -f v4l2 -i /dev/video0 -frames:v 1",
"maxStreams": 2,
"maxWidth": 640,
"maxHeight": 480,
"maxFPS": 10,
"vcodec": "h264_omx",
"debug": true,
"additionalCommandline": " ",
"audio": "2way -filter:a volume=12dB -f alsa hw:CARD=ALSA,0"
}
}
]
}
Clients (iPhones, iPad) all run iOS 12.0.1.
Fushicai USBTV007 Video Grabber [EasyCAP]
my doorbell hw is plain old analog video doorbell...
so video input is grabbed from /dev/video0, audio input is grabbed form (alsa) hw:CARD=usbtv, and audio ouput is (alsa) hw:CARD=ALSA,0 to the PI phone jack then connected to doorbell audio input
@llemtt
So, if audio is working on the ffmpeg stream, and with"audio": "2way" the intercom pops up without crashing, it would suffice to install libasound2, and set "audio": "2way -filter:a volume=12dB -f alsa hw:CARD=ALSA,0"?
Or is your setup actually hardwired?
@paqpaqpaq
In order to use alsa, ffmpeg must be compiled with libasound2-dev installed, so first install the lib then compile ffmpeg.
Next I suggest to try first that ffmpeg works by sending some audio file to the pi audio jack like this:
ffmpeg -i AUDIOFILE -f alsa hw:CARD=ALSA,0
If everything is ok then with my POC ffmpeg.js and this configuration:
"audio": "2way -f alsa hw:CARD=ALSA,0"
the intercom from the iPhone/iPad will be sent to the pi audio jack.
thanks.
So, there needs to be a hardwired connection between the audio jack from the pi to the cam, regardless?
@paqpaqpaq
I don't know, it depends on your cam!
My doorbell is an old "analog" type so I connected the audio jack to his "analog" audio input.
For instance I have an old version D-Link DCS-942L, but it supports 2way audio only through a proprietary audio format (ACAS) so it actually works only with their app...
If your cam accept an http or rtp or any "standard" input stream, you can put the necessary ffmpeg "output" configuration into "audio" property in place of "-f alsa hw:CARD=ALSA,0".
@llemtt I am trying to setup a video doorbell. Video works well, audio input works but it sounds a little fast than normal, and audio output (from iPhone/iPad to RPi) is not good, it sounds slow and very distorted. I have not idea where is the problem.
I am using a Raspberry Pi 3 Model B+, with its camera under /dev/video0, audio input with an USB device and audio output to RPi audio jack.
I installed [email protected] and replaced ffmpeg.js with your PoC file from here https://github.com/KhaosT/homebridge-camera-ffmpeg/issues/174.
My config.json is:
{
"bridge": {
"name": "DoorBell-Bridge",
"username": "AC:54:3D:B3:12:34",
"port": 51826,
"pin": "123-45-678"
},
"platforms": [
{
"platform": "Video-doorbell",
"cameras": [
{
"name": "doorbell",
"port": 5005,
"button": true,
"videoConfig": {
"source": "-f v4l2 -i /dev/video0 -f alsa -i hw:CARD=AK5371,0",
"stillImageSource": "-nostats -nostdin -timelimit 2 -analyzeduration 1 -f v4l2 -i /dev/video0 -frames:v 1",
"maxStreams": 2,
"maxWidth": 640,
"maxHeight": 480,
"maxFPS": 15,
"vcodec": "h264_omx",
"maxBitrate": 90,
"debug": true,
"additionalCommandline": " ",
"audio": "2way -f alsa hw:CARD=ALSA,0"
}
}
]
}
]
}
- My ffmpeg is:
ffmpeg version N-93099-g5c515b5f7d Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
configuration: --prefix=/usr/local --arch=armel --target-os=linux --enable-omx --enable-omx-rpi --enable-nonfree --enable-gpl --enable-libfdk-aac --enable-mmal --enable-libx264 --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable-demuxer=rtsp
Many thanks for your help.
@llemtt Im trying to get Audio Output working.
ffmpeg -i test.mp3 -f alsa hw:0,0 works fine.
But i got an Error when using Home App (Adress already in use)
Feb 11 01:40:19 debian homebridge[1653]: ffmpeg -nostats -nostdin -re -i rtsp://192.168.10.37:554/Streaming/channels/101 -map 0:0 -vcodec libx264 -pix_fmt yuv420p -r 10 -f rawvideo -loglevel verbose -vf scale=1280:720 -b:v 299k -bufsize 299k -maxrate 299k -payload_type 99 -ssrc 13533811 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params vPbTvvYUrwRrn1qHY/ByxUtpUzzPXZfrR5A8VMhM srtp://192.168.10.140:54220?rtcpport=54220\&localrtcpport=54220\&pkt_size=1316 -map 0:1 -acodec libfdk_aac -profile:a aac_eld -flags +global_header -f null -ar 16k -b:a 24k -bufsize 24k -ac 1 -payload_type 110 -ssrc 15163982 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params BNWdkG7KPZkEVCCfbN4W36jrS79j2155Y1SxUVwt srtp://127.0.0.1:59118?rtcpport=59118\&localrtcpport=9998\&pkt_size=188 Feb 11 01:40:19 debian homebridge[1653]: [Speaker] ffmpeg -v error -nostats -nostdin -max_ts_probe 0 -protocol_whitelist file,udp,rtp -i Tür-Cam_speaker.sdp -f speaker.mp4 Feb 11 01:40:19 debian homebridge[1653]: proxy listening 0.0.0.0:59118 Feb 11 01:40:19 debian homebridge[1653]: ffmpeg version N-93107-g6de396c216 Feb 11 01:40:19 debian homebridge[1653]: Copyright (c) 2000-2019 the FFmpeg developers Feb 11 01:40:19 debian homebridge[1653]: built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516 Feb 11 01:40:19 debian homebridge[1653]: configuration: --enable-gpl --enable-nonfree --enable-libfdk-aac --enable-libgsm --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-zlib --enable-postproc --enable-swscale --enable-pthreads --enable-libdc1394 --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb Feb 11 01:40:19 debian homebridge[1653]: libavutil 56. 26.100 / 56. 26.100 Feb 11 01:40:19 debian homebridge[1653]: libavcodec 58. 47.100 / 58. 47.100 Feb 11 01:40:19 debian homebridge[1653]: libavformat 58. 26.101 / 58. 26.101 Feb 11 01:40:19 debian homebridge[1653]: libavdevice 58. 6.101 / 58. 6.101 Feb 11 01:40:19 debian homebridge[1653]: libavfilter 7. 48.100 / 7. 48.100 Feb 11 01:40:19 debian homebridge[1653]: libswscale 5. 4.100 / 5. 4.100 Feb 11 01:40:19 debian homebridge[1653]: libswresample 3. 4.100 / 3. 4.100 Feb 11 01:40:19 debian homebridge[1653]: libpostproc 55. 4.100 / 55. 4.100 Feb 11 01:40:19 debian homebridge[1653]: [tcp @ 0x55668eafba80] Feb 11 01:40:19 debian homebridge[1653]: [Speaker] [udp @ 0x556bff622040] Feb 11 01:40:19 debian homebridge[1653]: [Speaker] bind failed: Address already in use Feb 11 01:40:19 debian homebridge[1653]: [Speaker] [udp @ 0x556bff6222c0] Feb 11 01:40:19 debian homebridge[1653]: [Speaker] bind failed: Address already in use Feb 11 01:40:19 debian homebridge[1653]: [Speaker] Tür-Cam_speaker.sdp: Invalid data found when processing input
Is there a Problem with : proxy listening 0.0.0.0:59118 ?
Greeds Dominik
I found my problem with distorted audio. RPi audio jack and camera don't work well when using kernel module to get /dev/video0. I disconnected camera, unloaded kernel module and deactivated in rasp-config; then I created a dummy video input, and now 2-way audio works. Actually, I am waiting for a new USB camera from Amazon. I hope with the new cam, all works fine.
@dosordie I suggest to change in config.json "Tür-Cam" name for other without any special character, like "mycam". Then, you can rename it into Home App.
@RavenSystem Changed the Name in Config.json. Same Problem
srtp://192.168.10.140:57575?rtcpport=57575&localrtcpport=57575&pkt_size=1316
In my network there is no device with the address 192.168.10.140 - Could this be a Problem? But I do not know where to change that
@dosordie
It's very strange because in my code the proxy is hard wired to listen on port 9999.
Anyway I never tried it on debian (pc?) so nodejs udp sockets can behave a little different there.
192.168.10.140 must be your iPhone/iPad/AppleTV address!
@llemtt Your PoC is amazing! Have you an improved version or are you working on it?
@llemtt Another question: is there any way to increase 10 seconds speaker timeout?
Thanks!
@llemtt Your PoC is amazing! Have you an improved version or are you working on it?
Many thanks. Sorry but nowadays I don't have time to work on it, I have my videodoorbell working so I will get back to it only if I would need to change/fix something.
Consider also that code is no longer up to date to latest version of ffmpeg-camera and requires some rework to eventually merge.
@llemtt Another question: is there any way to increase 10 seconds speaker timeout?
Not an easy way, that timeout (rtp/udp timeout) is hardcoded into ffmpeg and cannot be passed as a parameter so one can:
- modify own instance of ffmpeg
- ask ffmpeg maintainers to let timeout be passed as a parameter (I have already done that anyway... but don't think it will see any future soon)
- generate dummy rtcp packets in the proxy while the true ones are not received (just an idea)
@llemtt
Not an easy way, that timeout (rtp/udp timeout) is hardcoded into ffmpeg and cannot be passed as a parameter so one can:
- modify own instance of ffmpeg
- ask ffmpeg maintainers to let timeout be passed as a parameter (I have already done that anyway... but don't think it will see any future soon)
- generate dummy rtcp packets in the proxy while the true ones are not received (just an idea)
Do you know where is hardcoded into ffmpeg?
Update: https://trac.ffmpeg.org/ticket/2415
@llemtt
@dosordie
It's very strange because in my code the proxy is hard wired to listen on port 9999.
Anyway I never tried it on debian (pc?) so nodejs udp sockets can behave a little different there.
Is there a way to change the port (by variable or in the script File?) Yes it´s a Debian on a VM. Ok, the IP is from my iPhone...
@RavenSystem
Do you know where is hardcoded into ffmpeg?
#define READ_PACKET_TIMEOUT_S 10
in file rtsp.c
Update: https://trac.ffmpeg.org/ticket/2415
exactly! voted (+1) on that ticket
@dosordie
port used are 9998 and 9999, search ffmpeg.js for "999" you'll find all occurrences
port 9999 is used inside the .sdp speaker file that ffmpeg uses as the speaker input description
@llemtt I do not think the port is the problem, right? The error message is "bind failed: Address already in use" Can the IP? Address be set anywhere in config or ffmpeg.js?
@dosordie Actually is a port problem, systemwise you can have just one socket bound to the same port although, depending on OS, you can bound the same port on different IP addresses (for instance if you have more than one interface or an interface with multiple IPs).
I can think two things
- on your system port 9999 is already used by some other process, you can check with "netstat -n | grep 9999"; check also that you don't have other ffmpeg instances hanging around...
- your system consider ports less than 10000 (or some higher number) reserved, so doesn't let you use it
@llemtt I found the problem. It was "Webmin" which belonged to the port 10000. Probably that has the port 9999 reserved.
Now it works good. Top work! Thank you!
How could this work with a camera like the Axis 3343 which supports 2-way audio. From the documentation, it shows: QuickTimeTM supports G.711 and AAC audio encoding. The following paths can be used:
- rtsp://
/axis-media/media.amp I have been able to use applications on the ipad which support 2 way audio in the past. And now I only have 1 way audio from the camera, listening. Thank you guys for your work on this. Will be great to put together a real two-way audio/intercom system.
@llemtt Please, I hope you can help me. I have an USB camera that works under /dev/video0, an USB mic that works with ALSA, and the RPi audio jack to use the speaker. I am using second version of your working PoC, that I think is the last.
But I have 2 problems with audio: a) Audio from RPi to iPhone: The audio is heard with a little higher frequency. b) Audio from iPhone to RPi: The audio is heard with a big lower frequency.
I don't know if it's necessary to customize ALSA setup in any way, or do other things.
And another question: I only use [email protected], but I see that you use [email protected] too in your setup. Is it necessary?
Many thanks for your help!
@RavenSystem I remember that audio one octave higher or lower was something I experienced at the beginning but it disappeared once I moved on with configurations. I think it was related to an issue of 8 vs 16 bits pcm between ffmpeg and alsa.
I went directly to the hw card also because using default + alsa configuration wasn't completely ok.
What happens if you record the speaker audio on a file with "2way speaker.mp4" and play back to phone jack with ffmpeg? Is the pitch correct?
For my doorbell I use [email protected] but I was testing another two-way cam with [email protected], it's not necessary.
@llemtt I am going to do some records but, can you tell me how to manipulate hardware audio card directly, please?
@llemtt I don't see the 2way code getting activated with video-doorbell from ffmpeg.config. I also don't see anything to make outbound communication from the phone. Am I missing something?
I have this error please help.
[NULL @ 0x2c88330] Requested output format 'alsa' is not a suitable output format hw:CARD=ALSA,0: Invalid argument
pi@iDomoPi3Plus:~$ ffmpeg -i dingdong.mp3 -f alsa hw:CARD=ALSA,0
ffmpeg version N-93878-g81acc9adbf Copyright (c) 2000-2019 the FFmpeg developers built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516 configuration: --prefix=/usr/local --arch=armel --target-os=linux --enable-omx-rpi --enable-nonfree --enable-gpl --e nable-libfdk-aac --enable-mmal --enable-libx264 --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable- demuxer=rtsp libavutil 56. 28.100 / 56. 28.100 libavcodec 58. 52.101 / 58. 52.101 libavformat 58. 27.103 / 58. 27.103 libavdevice 58. 7.100 / 58. 7.100 libavfilter 7. 53.101 / 7. 53.101 libswscale 5. 4.101 / 5. 4.101 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100 [mp3 @ 0x2c85210] Skipping 417 bytes of junk at 45. [mp3 @ 0x2c85210] Estimating duration from bitrate, this may be inaccurate Input #0, mp3, from 'dingdong.mp3': Metadata: title : http://www.freesfx.co.uk Duration: 00:00:01.85, start: 0.000000, bitrate: 129 kb/s Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s [NULL @ 0x2c88330] Requested output format 'alsa' is not a suitable output format hw:CARD=ALSA,0: Invalid argument
