go2rtc icon indicating copy to clipboard operation
go2rtc copied to clipboard

Two way talk for Chinese version of TP link cameras

Open christaikobo opened this issue 7 months ago • 7 comments

There are a bunch of cheap IP cameras (TP link IPCXXXX) by TP link that are sold in China: https://www.tp-link.com.cn/list_2549.html

They use RTSP for video and audio streaming by default, and go2rtc picks up that without any issue.

But for two way talk, they have some proprietary protocols similar to that used in TAPO cameras, only difference is that it is wrapped inside RTSP instead of HTTP. Someone did a bit of packet capture and reverse engineering, and claimed that it was a RTSP MULTITRANS plugin. (https://iovxw.net/p/frigate/)

If anyone is interested in implementing it, I can provide a camera access for you, because I don't know how to do it.

Thank you!

christaikobo avatar May 07 '25 17:05 christaikobo

Hi Alex, @AlexxIT these cameras support RTSP and the send_only voice socket can be opened by MULTITRANS auth like this: https://github.com/bingooo/hass-tplink-ipc/blob/main/custom_components/tplink_ipc/talkback.py

    def _connect_and_auth(self):
        """Performs the full 3-step MULTITRANS handshake."""
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.settimeout(10)
        try:
            sock.connect((self._ip, 554))
            uri = f"rtsp://{self._ip}/multitrans"

            # Step 1: Get challenge
            req1 = (f"MULTITRANS {uri} RTSP/1.0\r\nCSeq: 0\r\n"
                    f"X-Client-UUID: {self._client_uuid}\r\n\r\n")
            sock.sendall(req1.encode())
            resp1 = sock.recv(2048).decode()
            if "401" not in resp1: raise ConnectionError("Failed to get auth challenge.")
            
            digest_line = [l for l in resp1.split('\r\n') if 'WWW-Authenticate: Digest' in l][0]
            realm = digest_line.split('realm="')[1].split('"')[0]
            nonce = digest_line.split('nonce="')[1].split('"')[0]
            
            # Step 2: Send auth
            response = self._calculate_digest(realm, nonce, "MULTITRANS", uri)
            auth_header = f'Digest username="{self._user}", realm="{realm}", nonce="{nonce}", uri="{uri}", response="{response}"'
            req2 = (f"MULTITRANS {uri} RTSP/1.0\r\nCSeq: 1\r\nAuthorization: {auth_header}\r\n"
                    f"X-Client-UUID: {self._client_uuid}\r\n\r\n")
            sock.sendall(req2.encode())
            resp2 = sock.recv(2048).decode()
            if "200 OK" not in resp2: raise ConnectionRefusedError("Authentication failed.")
            session_id = [l for l in resp2.split('\r\n') if 'Session:' in l][0].split(': ')[1].strip()

            # Step 3: Open talk channel
            payload = json.dumps({"type":"request","seq":0,"params":{"method":"get","talk":{"mode":"half_duplex"}}})
            req3 = (f"MULTITRANS {uri} RTSP/1.0\r\nCSeq: 2\r\nSession: {session_id}\r\n"
                    f"Content-Type: application/json\r\nContent-Length: {len(payload)}\r\n\r\n{payload}")
            sock.sendall(req3.encode())
            resp3 = sock.recv(2048).decode()
            if '"error_code":0' not in resp3: raise ConnectionError("Failed to open talkback channel.")
            
            _LOGGER.info("Successfully authenticated and talkback channel is open.")
            return sock
        except Exception as e:
            sock.close()
            _LOGGER.error(f"Handshake failed: {e}")
            return None

by the way, these camera have a web ui lisening on port 80, which have two kind of auth method, encrypt_type "3" (MD5 method) like VIGI, and encrypt_type "2" (RSA method)。

encrypt_type "3" MD5 method auth implement: https://github.com/bingooo/hass-tplink-ipc/blob/main/custom_components/tplink_ipc/api.py

encrypt_type "2" (RSA method) auth implement: https://github.com/iapYang/tplink_ipc_implement/blob/main/custom_components/tplink_ipc_implement/core.py

I have tested encrypt_type "2" loggin, and MULTITRANS method to open the socket, both works on my TP-LINK IPC485EP-AI, hope these infomation is useful!

forrestsocool avatar Oct 11 '25 02:10 forrestsocool

It's difficult to add support blindly without having such a camera.

AlexxIT avatar Oct 11 '25 07:10 AlexxIT

I can provide a camera. I can send a email with the access information if you want to take a look at it. It would be much appreciated.

On Sat, Oct 11, 2025, 04:55 Alex X @.***> wrote:

AlexxIT left a comment (AlexxIT/go2rtc#1724) https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3393037589

It's difficult to add support blindly without having such a camera.

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3393037589, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFY26LBKIMNGRJRECYN6RXL3XCZVNAVCNFSM6AAAAAB4UQMYCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJTGAZTONJYHE . You are receiving this because you authored the thread.Message ID: @.***>

christaikobo avatar Oct 11 '25 11:10 christaikobo

It's still okay to add support for regular video remotely. But adding two-way audio support is almost impossible. There's no feedback. I don't understand whether the code is working correctly or not.

AlexxIT avatar Oct 11 '25 12:10 AlexxIT

I can help with that, I can do 30 minute test sessions multiple times if you need. What kind of feedback are we looking for here? If it's simple "is there any sound coming out of the camera" I think it wont be too difficult.

On Sat, Oct 11, 2025, 09:47 Alex X @.***> wrote:

AlexxIT left a comment (AlexxIT/go2rtc#1724) https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3393285319

It's still okay to add support for regular video remotely. But adding two-way audio support is almost impossible. There's no feedback. I don't understand whether the code is working correctly or not.

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3393285319, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFY26LGBU53UL2S7DOKNA6T3XD35RAVCNFSM6AAAAAB4UQMYCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJTGI4DKMZRHE . You are receiving this because you authored the thread.Message ID: @.***>

christaikobo avatar Oct 11 '25 17:10 christaikobo

It's important to listen to what the camera is outputting. What's the latency? Are there any errors? I have an experience with adding remotely two way audio. It takes a lot of power, and the results are poor. It's better to spend this time and energy on more important tasks.

AlexxIT avatar Oct 12 '25 10:10 AlexxIT

Sure, if that's your experience then it's fair that you don't want to do it this way.

On Sun, Oct 12, 2025, 07:57 Alex X @.***> wrote:

AlexxIT left a comment (AlexxIT/go2rtc#1724) https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3394163225

It's important to listen to what the camera is outputting. What's the latency? Are there any errors? I have an experience with adding remotely two way audio. It takes a lot of power, and the results are poor. It's better to spend this time and energy on more important tasks.

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/1724#issuecomment-3394163225, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFY26LGNQJ47FPHTI2CXYTL3XIX2VAVCNFSM6AAAAAB4UQMYCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJUGE3DGMRSGU . You are receiving this because you authored the thread.Message ID: @.***>

christaikobo avatar Oct 12 '25 11:10 christaikobo

Hi @christaikobo , This https://github.com/AlexxIT/go2rtc/pull/1995 PR works on my TP-LINK IPC485EP-AI camera, you can have a test.

forrestsocool avatar Dec 22 '25 07:12 forrestsocool