HomeAssistant-Tapo-Control icon indicating copy to clipboard operation
HomeAssistant-Tapo-Control copied to clipboard

Published 20 hours ago • verified publisher icon JurajNyiri

Feature Request: Add multiple events detections

Open AndreiArdelean1 opened this issue 1 year ago • 46 comments

Is your feature request related to a problem? Please describe. I want to be able to detect multiple event types. I'm not interested in actually highlighting/retrieving the rectangle where this was detected on the camera, just the detection event.

Describe the solution you'd like I want to be able to detect and distinguish between:

  • Motion
  • AI: Person, Pet, Vehicle, Baby Crying, Glass Break, Dog Barks, Cat Meows
  • Line-Crossing
  • Camera Tampering

These notifications are sent to the native app.

Describe alternatives you've considered

  1. Create a different sensor for each event type. This has the benefit of also tracking (either now, or maybe in future updates of the camera) the duration that the event was on.
  2. Add attributes to the motion detection sensor binary_sensor that indicate the type of the event

A clear and concise description of any alternative solutions or features you've considered.

Additional context These features are available on 3MP+ cameras for free, and on 1080p cameras using the paid subscription

Here is the response from GetEventProperties of the ONVIF endpoint on Tapo C225 camera:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:chan="http://schemas.microsoft.com/ws/2005/02/duplex" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:xmime="http://tempuri.org/xmime.xsd" xmlns:xop="http://www.w3.org/2004/08/xop/include" xmlns:wsrfbf="http://docs.oasis-open.org/wsrf/bf-2" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsrfr="http://docs.oasis-open.org/wsrf/r-2" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:ter="http://www.onvif.org/ver10/error" xmlns:tns1="http://www.onvif.org/ver10/topics" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:trp="http://www.onvif.org/ver10/replay/wsdl" xmlns:tan="http://www.onvif.org/ver20/analytics/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:hikwsd="http://www.onvifext.com/onvif/ext/ver10/wsdl" xmlns:hikxsd="http://www.onvifext.com/onvif/ext/ver10/schema">
    <SOAP-ENV:Header>
        <wsa5:Action SOAP-ENV:mustUnderstand="true">http://www.onvif.org/ver10/events/wsdl/EventPortType/GetEventPropertiesResponse</wsa5:Action>
    </SOAP-ENV:Header>
    <SOAP-ENV:Body>
        <tev:GetEventPropertiesResponse>
            <tev:TopicNamespaceLocation>http://www.onvif.org/onvif/ver10/topics/topicns.xml</tev:TopicNamespaceLocation>
            <wsnt:FixedTopicSet>true</wsnt:FixedTopicSet>
            <wstop:TopicSet>
                <tns1:RuleEngine>
                    <CellMotionDetector>
                        <Motion wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsMotion"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </Motion>
                    </CellMotionDetector>
                    <TamperDetector>
                        <Tamper wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsTamper"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </Tamper>
                    </TamperDetector>
                    <LineCrossDetector>
                        <LineCross wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsLineCross"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </LineCross>
                    </LineCrossDetector>
                    <PeopleDetector>
                        <People wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsPeople"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </People>
                    </PeopleDetector>
                    <TPSmartEventDetector>
                        <TPSmartEvent wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsTPSmartEvent"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </TPSmartEvent>
                    </TPSmartEventDetector>
                </tns1:RuleEngine>
            </wstop:TopicSet>
            <wsnt:TopicExpressionDialect>http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet</wsnt:TopicExpressionDialect>
            <wsnt:TopicExpressionDialect>http://docs.oasis-open.org/wsn/t-1/TopicExpression/Concrete</wsnt:TopicExpressionDialect>
            <tev:MessageContentFilterDialect>http://www.onvif.org/ver10/tev/messageContentFilter/ItemFilter</tev:MessageContentFilterDialect>
            <tev:MessageContentSchemaLocation>http://www.onvif.org/onvif/ver10/schema/onvif.xsd</tev:MessageContentSchemaLocation>
        </tev:GetEventPropertiesResponse>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Currently, motion detection doesn't work on Tapo C225 #303, but this feature would be nice to have once it is working.

AndreiArdelean1 avatar Mar 30 '23 07:03 AndreiArdelean1

This integration reuses onvif from Home Assistant. Adding detectors for these would require to expand on it or handlers to be added into https://github.com/home-assistant/core/blob/dev/homeassistant/components/onvif/parsers.py .

JurajNyiri avatar Mar 31 '23 13:03 JurajNyiri

  1. Install https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif
  2. You should see a lot more warning messages now on onvif events trigger
  3. Unsupported entities should come up as "No registered handler for event from" or "Unable to parse event from"

Please report them here.

I tested on C200 with my Person Detection enabled and I have not received any events...

JurajNyiri avatar Mar 31 '23 14:03 JurajNyiri

I would like to help as I need my tapo cameras to trigger automations in HA when a person is detected. Until now the only way I could achieve that is to use Alexa between the cameras and HA. With this solution the cameras need to access internet which is not ideal. Using the video stream to detect people on another system is also not a solution because it uses the WiFi so much that it interferes with my Philips Hue remotes.

Could you please tell me how to install https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif ?

NEVdataDyne avatar Mar 31 '23 15:03 NEVdataDyne

  1. Download https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/archive/refs/heads/expand_onvif.zip
  2. Follow https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif#installation

JurajNyiri avatar Mar 31 '23 17:03 JurajNyiri

I downloaded the zip I updated the Tapo custom component to 5.0 Then I replaced all the files custom_components\tapo_control by the ones from the zip using samba share and restarted HA I didn't see any new entities so I installed the onvif integration and connected it to one of my C225 camera I didn't see any new entities either (besides the onvif related entities that I was expecting) In the logs there doesn't seem to be a message like "No registered handler for event from" or "Unable to parse event from"

I am not sure what I should do?

NEVdataDyne avatar Mar 31 '23 20:03 NEVdataDyne

The code in the branch above doesnt have any new entities. However, it will catch any previously uncought events with an error message. If you are not seeing any other events then it looks like the camera is not issueing those events. I will do some more testing later with the subscription.

JurajNyiri avatar Mar 31 '23 21:03 JurajNyiri

I do have many errors and warnings from the Tapo integrations. Three of them occur very often (several occurence each minute) I don't know if they are related to unsupported events but the warnings don't provide much info. They occur even when person and movement detection are turned off but there may be other unsupported events, if these are what you are looking for I don't know how to filter them for the specific events that interest us or to make the warning messages more usefull?

Warning 1 This error originated from a custom integration.

Logger: custom_components.tapo_control Source: custom_components/tapo_control/utils.py:561 Integration: Tapo: Cameras Control (documentation, issues) First occurred: 31 March 2023 at 22:06:48 (22736 occurrences) Last logged: 13:37:12

Setting up events...

Warning 2 This error originated from a custom integration.

Logger: custom_components.tapo_control Source: custom_components/tapo_control/utils.py:559 Integration: Tapo: Cameras Control (documentation, issues) First occurred: 31 March 2023 at 22:06:48 (22736 occurrences) Last logged: 13:37:12

setupEvents - entry

Error 1 This error originated from a custom integration.

Logger: custom_components.tapo_control Source: components/onvif/event.py:82 Integration: Tapo: Cameras Control (documentation, issues) First occurred: 31 March 2023 at 22:06:38 (22741 occurrences) Last logged: 13:37:11

Unexpected error fetching Tapo resource status data: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.132:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error') Unexpected error fetching Tapo resource status data: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.25:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZeHTTP/1.1 500 Internal Server Error') Unexpected error fetching Tapo resource status data: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.129:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error') Unexpected error fetching Tapo resource status data: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.140:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error') Unexpected error fetching Tapo resource status data: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.30:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZeHTTP/1.1 500 Internal Server Error') Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions yield File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 188, in _receive_event event = self._h11_state.next_event() File "/usr/local/lib/python3.10/site-packages/h11/_connection.py", line 487, in next_event exc._reraise_as_remote_protocol_error() File "/usr/local/lib/python3.10/site-packages/h11/_util.py", line 77, in _reraise_as_remote_protocol_error raise self File "/usr/local/lib/python3.10/site-packages/h11/_connection.py", line 469, in next_event event = self._extract_next_receive_event() File "/usr/local/lib/python3.10/site-packages/h11/_connection.py", line 411, in _extract_next_receive_event event = self._reader(self._receive_buffer) File "/usr/local/lib/python3.10/site-packages/h11/_readers.py", line 104, in maybe_read_from_SEND_RESPONSE_server matches = validate(status_line_re, lines[0], "illegal status line: {!r}", lines[0]) File "/usr/local/lib/python3.10/site-packages/h11/_util.py", line 91, in validate raise LocalProtocolError(msg) h11._util.RemoteProtocolError: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.135:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions yield File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request raise exc File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request response = await connection.handle_async_request(request) File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request return await self._connection.handle_async_request(request) File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 112, in handle_async_request raise exc File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 91, in handle_async_request ) = await self._receive_response_headers(**kwargs) File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_headers event = await self._receive_event(timeout=timeout) File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 187, in _receive_event with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}): File "/usr/local/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) httpcore.RemoteProtocolError: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.135:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 239, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 195, in _async_update_data return await self.update_method() File "/config/custom_components/tapo_control/init.py", line 234, in async_update_data await setupOnvif(hass, entry) File "/config/custom_components/tapo_control/utils.py", line 553, in setupOnvif hass.data[DOMAIN][entry.entry_id]["eventsSetup"] = await setupEvents( File "/config/custom_components/tapo_control/utils.py", line 563, in setupEvents if await events.async_start(): File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 82, in async_start if await self.device.create_pullpoint_subscription(): File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 350, in create_pullpoint_subscription pullpoint = await events.CreatePullPointSubscription() File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in call return await self._proxy._binding.send_async( File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async response = await client.transport.post_xml( File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml response = await self.post(address, message, headers) File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post response = await self.client.post( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post return await self.request( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send response = await self._send_handling_auth( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth response = await self._send_handling_redirects( File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects response = await self._send_single_request(request) File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request response = await transport.handle_async_request(request) File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request with map_httpcore_exceptions(): File "/usr/local/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.RemoteProtocolError: illegal status line: bytearray(b'POST /onvif/service\x00HTTP/1.1\x00\x00Host\x00 192.168.50.135:2020\x00\x00Accept\x00 /\x00\x00Accept-Encoding\x00 gzip, deflate, br\x00\x00Connection\x00 keep-alive\x00\x00User-Agent\x00 ZHTTP/1.1 500 Internal Server Error')

NEVdataDyne avatar Apr 01 '23 11:04 NEVdataDyne

@NEVdataDyne It looks like you are having the issue #303 (the last error in your comment). Is your motion sensor (classic one) working/triggered in Homeassistant?

@JurajNyiri I'm not seeing any of the errors you mentioned. Could it be that the integration needs to subscribe explicitly for certain events? Something like "on event PERSON_DETECTED post to WEBHOOK".

Also, because of #303 I'm having issues at the moment. I'm changing the file manually inside onvif-zeep-async but it sometimes gets overwritten or some other issue, but the motion detection (classic one) is not very stable for me. This could be the cause I'm not getting any of the new errors.

AndreiArdelean1 avatar Apr 01 '23 11:04 AndreiArdelean1

@AndreiArdelean1 No motion sensor doesn't seem to work. It doesn't seem to work even with my C110 camera so I am not really sure if I am doing somethng wrong. (I have C225, C320WS and one C110 camera)

NEVdataDyne avatar Apr 01 '23 15:04 NEVdataDyne

I've gotten the integration to work by modifying tapo_control to use a local version of onvif-zeep-async and only motion events are triggered. I've also verified this using Postman and observing the sent events.

Also, the ONVIF integration declares CellMotionDetector, TamperDetector, LineCrossDetector, PeopleDetector, TPSmartEventDetector. All of this should be just boolean values. But, I've only been able to trigger only CellMotionDetector. Since the camera supports other types of detection events/notifications (like pet detection, glass break, vehicle, etc., within the official app) + boundary boxes of the detection inside the video feed, (apparently) are not exposed through ONVIF, I'm guessing the camera uses some Tapo specific API for this. The TamperDetector, LineCrossDetector, and PeopleDetector are most likely discontinued from the ONVIF API or something else is going on.

I'll investigate this further and post updates if I find something else.

AndreiArdelean1 avatar Apr 02 '23 11:04 AndreiArdelean1

I didn't manage to find a way of getting the special detection events, but the following will help with integrating the configuration of these events (enabling and sensitivity) into HA and pytapo.

Here is the request the app makes on start: {"method":"multipleRequest","params":{"requests":[{"method":"getDetectionConfig","params":{"motion_detection":{"name":"motion_det"}}},{"method":"getDetectionRegion","params":{"motion_detection":{"table":["region_info"]}}},{"method":"getBCDConfig","params":{"sound_detection":{"name":["bcd"]}}},{"method":"getPersonDetectionConfig","params":{"people_detection":{"name":["detection"]}}},{"method":"getVehicleDetectionConfig","params":{"vehicle_detection":{"name":["detection"]}}},{"method":"getPetDetectionConfig","params":{"pet_detection":{"name":["detection"]}}},{"method":"getBarkDetectionConfig","params":{"bark_detection":{"name":["detection"]}}},{"method":"getMeowDetectionConfig","params":{"meow_detection":{"name":["detection"]}}},{"method":"getGlassDetectionConfig","params":{"glass_detection":{"name":["detection"]}}},{"method":"getTargetTrackConfig","params":{"target_track":{"name":["target_track_info"]}}},{"method":"getAlertConfig","params":{"msg_alarm":{"name":["chn1_msg_alarm_info","capability"]}}},{"method":"getAlertPlan","params":{"msg_alarm_plan":{"name":"chn1_msg_alarm_plan"}}},{"method":"getAlertTypeList","params":{"msg_alarm":{"name":"alert_type"}}},{"method":"getAlertEventType","params":{"msg_alarm":{"table":["msg_alarm_type"]}}},{"method":"getAlertConfig","params":{"msg_alarm":{"table":["usr_def_audio"]}}},{"method":"getMsgPushConfig","params":{"msg_push":{"name":"chn1_msg_push_info"}}},{"method":"getMsgPushPlan","params":{"msg_push_plan":{"name":"chn1_msg_push_plan"}}},{"method":"getTamperDetectionConfig","params":{"tamper_detection":{"name":"tamper_det"}}},{"method":"getLinecrossingDetectionConfig","params":{"linecrossing_detection":{"name":["detection","arming_schedule"]}}},{"method":"getLinecrossingDetectionRegion","params":{"linecrossing_detection":{"table":["region_info"]}}},{"method":"getMsgPushEventList","params":{"msg_push":{"table":["msg_push_event"]}}}]}}

A few examples of setting detection configs:

  • baby cry {"method":"multipleRequest","params":{"requests":[{"method":"setBCDConfig","params":{"sound_detection":{"bcd":{"enabled":"on","sensitivity":"medium"}}}}]}}

  • person {"method":"multipleRequest","params":{"requests":[{"method":"setPersonDetectionConfig","params":{"people_detection":{"detection":{"enabled":"off","sensitivity":"60"}}}}]}}

  • pet {"method":"multipleRequest","params":{"requests":[{"method":"setPetDetectionConfig","params":{"pet_detection":{"detection":{"enabled":"off","sensitivity":"60"}}}}]}}

  • bark {"method":"multipleRequest","params":{"requests":[{"method":"setBarkDetectionConfig","params":{"bark_detection":{"detection":{"enabled":"on","sensitivity":"50"}}}}]}}

  • glass break {"method":"multipleRequest","params":{"requests":[{"method":"setGlassDetectionConfig","params":{"glass_detection":{"detection":{"enabled":"on","sensitivity":"60"}}}}]}}

  • tamper {"method":"multipleRequest","params":{"requests":[{"method":"setTamperDetectionConfig","params":{"tamper_detection":{"tamper_det":{"enabled":"on","sensitivity":"medium"}}}}]}}

AndreiArdelean1 avatar Apr 03 '23 08:04 AndreiArdelean1

Updated the code in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif

  1. Download https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/archive/refs/heads/expand_onvif.zip
  2. Follow https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif#installation

You should now see:

  1. Warn message for "async_parse_messages" every time any event is detected
  2. Warn Message Found Event when it found an event it recognizes
  3. Error message "No registered handler for event from" or "Unable to parse event from" when event is found which is not recognized
2023-04-03 08:49:23.920 WARNING (MainThread) [custom_components.tapo_control] async_parse_messages

2023-04-03 08:49:23.921 WARNING (MainThread) [custom_components.tapo_control] Found event ba835a1edb0d0cdf5bcbafab8b732021_tapo_events: {

    'SubscriptionReference': None,

    'Topic': {

        '_value_1': 'tns1:RuleEngine/CellMotionDetector/Motion',

        'Dialect': 'http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet',

        '_attr_1': {

    }

    },

    'ProducerReference': None,

    'Message': {

        '_value_1': {

            'Source': {

                'SimpleItem': [

                    {

                        'Name': 'VideoSourceConfigurationToken',

                        'Value': 'vsconf'

                    },

                    {

                        'Name': 'VideoAnalyticsConfigurationToken',

                        'Value': 'VideoAnalyticsToken'

                    },

                    {

                        'Name': 'Rule',

                        'Value': 'MyMotionDetectorRule'

                    }

                ],

                'ElementItem': [],

                'Extension': None,

                '_attr_1': None

            },

            'Key': None,

            'Data': {

                'SimpleItem': [

                    {

                        'Name': 'IsMotion',

                        'Value': 'true'

                    }

                ],

                'ElementItem': [],

                'Extension': None,

                '_attr_1': None

            },

            'Extension': None,

            'UtcTime': datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<isodate.tzinfo.Utc object at 0x7f663048c580>),

            'PropertyOperation': 'Changed',

            '_attr_1': {

        }

        }

    }

}

JurajNyiri avatar Apr 03 '23 08:04 JurajNyiri

I've already done something similar, but the only thing that changed was the Data.SimpleItem.Value (+ UtcTime). No new event was received by tapo_control, even though the notification was sent to the app. Probably the camera doesn't sent any of the new events using ONVIF.

AndreiArdelean1 avatar Apr 03 '23 09:04 AndreiArdelean1

I have added to main branch (not yet released to pypi):

  • get/set VehicleDetection
  • get/set Cruise
  • get/set MeowDetection
  • get/set BarkDetection
  • get/set PetDetection
  • get/set BabyCryDetection
  • get/set TamperDetection
  • get/set GlassBreakDetection

I was not able to test:

setBarkDetection setPetDetection setGlassBreakDetection setMeowDetection setVehicleDetection getBarkDetection getPetDetection getGlassBreakDetection getMeowDetection getVehicleDetection getCruise

@AndreiArdelean1 could you please test above?

Edit: Released as pytapo 3.1.8.

JurajNyiri avatar Apr 03 '23 11:04 JurajNyiri

@AndreiArdelean1 now released in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/releases/tag/5.1.0.beta.1 Please test above, my devices do not support it.

JurajNyiri avatar Apr 03 '23 13:04 JurajNyiri

They all work except for getCruise. getCruise always is {'patrol': {'patrol': {'action': 'idle'}}} regardless if it is idle or doing the cruise. The HA integrations also works. 🎉

AndreiArdelean1 avatar Apr 03 '23 17:04 AndreiArdelean1

I've been investigating this issue and here is what I've found so far:

ONVIF:

  • the ONVIF endpoint declares IsMotion, IsTamper, IsLineCross, IsPeople, IsTPSmartEvent
  • for IsTamper, IsLineCross I'm not sure if they are triggered (I don't remember if I tested this), but IsPeople is not triggered
  • Pet, Vehicle, Baby Crying, Glass Break, Dog Barks, Cat Meows, are not even declared in the ONVIF endpoint

UDP:

  • There aren't any multicast packets sent when an event is triggered
  • The app has the option to highlight people, pets, etc. in the live stream. The only place where this could take place (from the network packets received by the app) is inside the video/mp2t in the live stream from port 8800 (https://md.depau.eu/s/r1Ys_oWoP#Live-streaming, https://drmnsamoliu.github.io/video.html). There are no additional application/json received besides the initial ones (informing of the privacy mode overlay). I've not been able to decode the sent JSON packets, only the received ones, as they are encrypted. Most likely the highlighted zones are already included in the video.

API:

  • the App has a feature to view recordings where it also indicates the type of event triggered.
  • when the app enters that screen, here are the calls made:
{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230431","start_date":"20230401"}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"getSdCardStatus","params":{"harddisk_manage":{"table":["hd_info"]}}},{"method":"getCircularRecordingConfig","params":{"harddisk_manage":{"name":"harddisk"}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"getClockStatus","params":{"system":{"name":"clock_status"}}},{"method":"getDstRule","params":{"system":{"name":"dst"}}},{"method":"getTimezone","params":{"system":{"name":["basic"]}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230331","start_date":"20230301"}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchVideoWithUTC","params":{"playback":{"search_video_with_utc":{"id":1,"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDetectionList","params":{"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230231","start_date":"20230201"}}}}]}}

What can be used, for event detection is the call:

{"method":"searchDetectionList","params":{"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}

and returning:

{'playback': {'snapshot_enable': True, 'search_detection_list': [{'start_time': 1680725544, 'end_time': 1680725662, 'alarm_type': 6}, {'start_time': 1680725693, 'end_time': 1680725802, 'alarm_type': 6}, {'start_time': 1680725826, 'end_time': 1680725856, 'alarm_type': 6}, {'start_time': 1680725917, 'end_time': 1680725947, 'alarm_type': 6}], 'total_num': 4}}

A few things to note here:

  • the end time extends if multiple events are triggered in quick succession
  • the alarm type also updates. People, pet detection is preferred over motion events.
  • the start_time and end_time define a range and the camera returns the events that have at least 1s within that range (event.start_time can be before request.start_time, and the same for end_time)

I've written this python script to get the events from the last 10 min to test it:

secondsFrom1970 = tapo.executeFunction(
            "getClockStatus", {"system":{"name":"clock_status"}}
          )["system"]["clock_status"]["seconds_from_1970"]
print(secondsFrom1970)
searchStart = secondsFrom1970 - (10 * 60) # 10 min
searchEnd = secondsFrom1970 + 10
print(tapo.executeFunction(
  "searchDetectionList", {"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":searchStart,"end_time":searchEnd,"end_index":99}}}
))

Maybe this method of detection motion could be used to detect motion on https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/issues/273. This could be done by:

  1. making the above call every 0.5-1s and getting the last event (named here lastEvent) from the list and later storing it to savedLastEvent.
  2. if the start_time of lastEvent and savedLastEvent are not the same, a new event was triggered
  3. if the alarm_type of lastEvent and savedLastEvent are not the same, a new event was triggered
  4. if savedLastEvent has completed, a new event was triggered
  5. if secondsFrom1970 is within 30s (this is the default/minimum duration of an event) or 10s from the end_time, the event could still be active, otherwise the lastEvent can be marked as completed

Instead of making a call periodically, the ONVIF motion event could be used to trigger the call to searchDetectionList. BUT, the motion event is not triggered if only an AI event is detected. If the motion sensitivity is set high enough, the 2 events would be triggered in close succession and this would probably work.

Probably the best solution would be a combination of making a call periodically and watching for ONVIF motion events.

Other possible methods not yet discovered/implemented on the camera:

  • adding the missing events to ONVIF
  • the camera to emit a multicast broadcast (preferably encrypted) whenever something changes or just on events triggered
  • a way to register a webhook from the camera API for this
  • a way to register a webhook from the Tapo server to HA. Something similar to https://ifttt.com/tplink_tapo/details, but from what I could find, the API is only available to select developers.
  • opening a socket connection to the camera for events (not likely to be done in the future, or to already exist)
  • intercept the calls made by the camera to Tapo servers for notification (if possible, too complicated for the average user for the purpose of this integration)
  • register webhooks with IFTTT (tested and working). The only problem here is that the free plan only allows 5 automations.
  • register webhooks with Alexa

AndreiArdelean1 avatar Apr 07 '23 07:04 AndreiArdelean1

Maybe in "Other possible methods not yet discovered/implemented" you can add "intercept the calls made by the camera to Alexa servers for notification" (for people detection). For some reason Alex seems to be the only service other than Tapo to get notified when a person is detected (and this is the only event Alexa can get).

NEVdataDyne avatar Apr 07 '23 13:04 NEVdataDyne

I've written this script (attached) to pull events from the camera. It works really well. I'd implement a proof of concept for Tapo-Control, but I'm not very familiar with HomeAssistant integrations. Maybe this method could be used alogside the existing ONVIF events for detecting motion. Notes: The 2 files need to be under the same directory. Archive.zip

AndreiArdelean1 avatar Apr 13 '23 07:04 AndreiArdelean1

Thank you @AndreiArdelean1 . I played with the script, it had a bug where it was calculating incorrect start time for cameras in different timezone to UTC-0. I adjusted a couple of things and released new function getEvents to pytapo 3.1.11.

I am testing with following script:

from pytapo import Tapo
import time
from datetime import datetime
import os

# Camera
user = os.environ.get("user")  # admin user
password = os.environ.get("password")  # cloud password
host = os.environ.get("ip")  # ip of the camera, example: 192.168.1.52

tapo = Tapo(host, user, password, password)

epoch_time = int(time.time())
secondsFrom1970 = epoch_time

minSearchStart = secondsFrom1970 - (60 * 60)

searchStart = minSearchStart # can be used as opt parameter in tapo.getEvents()
searchEnd = secondsFrom1970 + 60  # maybe add a few seconds? # can be used as opt parameter in tapo.getEvents()

while True:
    events = tapo.getEvents()
    if events:
        lastEvent = events[-1]

        newLastEventDictionary = {
            'startTime': lastEvent["start_time"],
            'endTime': lastEvent["end_time"],
            'alarmType': lastEvent["alarm_type"],
        }
        print(datetime.now())
        print(lastEvent)
        print(datetime.fromtimestamp(newLastEventDictionary['startTime']))
        print(datetime.fromtimestamp(newLastEventDictionary['endTime']))
        print("")
    else:
        print("none")
    time.sleep(1)

I tested on 2 C200s. One HW V1.0 and another one 3.0. The 3.0 also has tapo cloud subscription. I found that:

  • Events are not reported reliably, even if they are visible in the timeline of recordings.
  • Events are reported with start time in the future

I tested also with your script only to verify I haven't made any mistakes when writing the new function.

Could you try it and let me know if it works with your cameras?

JurajNyiri avatar Apr 13 '23 10:04 JurajNyiri

I get something like:

2023-04-13 15:57:14.501225
{'start_time': 1681390629, 'end_time': 1681390659, 'alarm_type': 2, 'startRelative': 4.0, 'endRelative': -26.0}
2023-04-13 15:57:09
2023-04-13 15:57:39
  • start time is always in the past (startRelative > 0)
  • for me, the events are received reliably. An event being a change in either start_time, end_time, or alarm_type of the last event

What I've noticed, if multiple sources (script and Tapo app) are requesting the events at once, one of them could return an empty list.

AndreiArdelean1 avatar Apr 13 '23 13:04 AndreiArdelean1

Off-topic, but I'm not sure where I should post this: All the possible values for alarm_type from searchDetectionList:

allKnownAlarmTypes = {
    1: "Timing",
    2: "Motion",
    3: "TAMPER",
    4: "Line Crossing",
    5: "Area Intrusion", 
    6: "Person",
    7: "Baby Cry",
    8: "Vehicle",
    9: "Pet",
    10: "Ring Alarm",
    11: "Dog Bark",
    12: "Cat Meow",
    13: "Alarm Glass",
    14: "Alarm Smoke",
    15: "Deliver Package",
    16: "Pickup Package",
    17: "Dollbell Ring Missed",
    18: "Dollbell Ring Answered",
    19: "Anti Theft",
}

I'm not sure which work, which don't, or what they mean. I've extracted them from the Android app. The raw values:

1: TIMING 
2: MOTION 
3: TAMPER 
4: LINECROSS 
5: AREAINTRUSION 
6: HUMAN 
7: BABYCRY 
8: VEHICLE 
9: PET 
10: RINGALARM 
11: ALARMBARK 
12: ALARMMEOW 
13: ALARMGLASS 
14: ALARMSMOKE 
15: DELIVERPACKAGE 
16: PICKUPPACKAGE 
17: DOLLBELLRINGMISSED 
18: DOLLBELLRINGANSWERED 
19: ANTITHEFT

AndreiArdelean1 avatar Apr 14 '23 15:04 AndreiArdelean1

@AndreiArdelean1 could you please test this script with the pytapo 3.1.13 to verify it still works? I think I managed to fix the timezone issue and it seems to be working fairly well now for me.

JurajNyiri avatar May 04 '23 14:05 JurajNyiri

It works!

AndreiArdelean1 avatar May 05 '23 06:05 AndreiArdelean1

When I try to run the script above on the latest python docker image I get the following error: File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='192.168.1.81', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)')))

Do you have a tip? Maybe it has something todo with the IP not being a hostname? Is there a way to set the request to insecure?

fightforlife avatar May 07 '23 12:05 fightforlife

@AndreiArdelean1 Just to clarify, you were able to extract the AI detected events (person, pet, vehicle, baby crying, etc.)? I was playing around for a minute with the getEvents function, but couldn't get it to work. Just wanted to make sure it would be worth the time to actually dive in... Thank you!

rmtrane avatar Jun 06 '23 16:06 rmtrane

Hey, yes, it is working. It did work for motion, person, and pet, but it wasn't working for tamper detection (Or I didn't manage to get it in my testing as that type of detection was a bit hard to trigger). Sounds detection (meow, bark, baby, glass break) I couldn't trigger the actual event during testing so I can't say if it is working. But, if an event is shown with the correct icon in the playback list in the official app, then you should also be able to detect it with this function.

You could:

  1. every x seconds request the most recent events. The downside is that it introduces a load on the camera, and since the camera has limited resources (CPU, memory, etc.) the official app may become unresponsive, and it may cause other unknown issues
  2. when an event is detected with ONVIF, wait for 1s (or a smaller amount of time to make sure the event registers) and request the latest event from the camera to get the actual event type.

I've only tested method 1. When investigating I had some issues with ONVIF so I couldn't test method 2.

AndreiArdelean1 avatar Jun 07 '23 16:06 AndreiArdelean1

Thank you for letting me know! I'm mainly interested in the sounds detection, so might try to get this to work.

(In case you're interested, I have had success in triggering the events resulting in notifications through the Tapo app by simply using youtube videos of baby crying, dog barking, etc.)

@JurajNyiri Any plans to implement something like this in the Home Assistant integration? I will probably try to do something ad-hoc, but would be really cool to have this in the integration 😊

rmtrane avatar Jun 07 '23 17:06 rmtrane

I’ve read this conversation with much interest, but didn’t understand what is actually possible now within the context of the integration. I own a Tapo C310 and am interested in the person detection. How should I proceed to integrate it?

cruckfi avatar Jul 27 '23 08:07 cruckfi

@cruckfi it is not currently available inside the integration at all. What you could do today, is write a custom script hitting the camera and sending the information to HA, for example through REST API.

I will eventually add this to the integration but I do not have any estimate as of today.

JurajNyiri avatar Jul 27 '23 09:07 JurajNyiri