core
core copied to clipboard
home-assistant
Invalid ONVIF SubscriptionManager RenewRequest - python-zeep WSA bug
The problem
The problem is missing addressing in the SubscriptionManager/RenewRequest.
The Header contains only the Security element.
Action, To and MessageID are missing in the Header of the Renew request.
As the result a lot of ONVIF cameras (XiongMai, Wanscam and other) response 400 (Bad Request) to RenewRequest.
It is impossible to add/setup these cameras at all.
You can see different errors in logs, but the reason is the same.
My patch to python-zeep which fixed this issue was reverted. So it must be fixed in Home Assistant code.
Environment
- Home Assistant Core release with the issue: 2021.1.5
- Last working Home Assistant Core release (if known): never
- Operating environment (OS/Container/Supervised/Core): Supervised
- Integration causing this issue: ONVIF
- Link to integration documentation on our website: https://www.home-assistant.io/integrations/onvif
Problem-relevant configuration.yaml
logger:
default: info
logs:
homeassistant.components.onvif: debug
Traceback/Error logs
2021-01-25 03:57:41 DEBUG (MainThread) [homeassistant.components.onvif] Setting up the ONVIF device management service
2021-01-25 03:57:41 DEBUG (MainThread) [homeassistant.components.onvif] Retrieving current device date/time
2021-01-25 03:57:41 DEBUG (MainThread) [homeassistant.components.onvif] Device date/time: 2021-01-24 23:57:42+00:00 | System date/time: 2021-01-24 23:57:41.482773+00:00
2021-01-25 03:57:44 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Cam1 - xx:xx:xx:xx:xx:xx for onvif
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 236, in async_setup
result = await component.async_setup_entry(hass, self) # type: ignore
File "/usr/src/homeassistant/homeassistant/components/onvif/__init__.py", line 77, in async_setup_entry
if not await device.async_setup():
File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 105, in async_setup
self.capabilities = await self.async_get_capabilities()
File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 252, in async_get_capabilities
pullpoint = await self.events.async_start()
File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 83, in async_start
await self.async_renew()
File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 154, in async_renew
await self._subscription.Renew(termination_time)
File "/usr/local/lib/python3.8/site-packages/zeep/proxy.py", line 64, in __call__
return await self._proxy._binding.send_async(
File "/usr/local/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 164, in send_async
return self.process_reply(client, operation_obj, response)
File "/usr/local/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 229, in process_reply
return self.process_error(doc, operation)
File "/usr/local/lib/python3.8/site-packages/zeep/wsdl/bindings/soap.py", line 381, in process_error
subcode_qname = as_qname(
File "/usr/local/lib/python3.8/site-packages/zeep/utils.py", line 36, in as_qname
raise XMLParseError("No namespace defined for %r (%r)" % (prefix, value))
zeep.exceptions.XMLParseError: No namespace defined for 'ter' ('ter:InvalidArgVal')
Additional information
![probot-home-assistant[bot] avatar](https://avatars.githubusercontent.com/in/36093?v=4 "Published by a pub.dev verified publisher"){kind=small}
Here is the problem request:
POST /onvif/events_service HTTP/1.1
Host: 192.168.3.208
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
User-Agent: Zeep/4.0.0 (www.python-zeep.org)
SOAPAction: "http://docs.oasis-open.org/wsn/bw-2/SubscriptionManager/RenewRequest"
Content-Type: application/soap+xml; charset=utf-8; action="http://docs.oasis-open.org/wsn/bw-2/SubscriptionManager/RenewRequest"
Content-Length: 1007
<?xml version='1.0' encoding='utf-8'?>
<soap-env:Envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
<soap-env:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken>
<wsse:Username>admin</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">XXX</wsse:Password>
<wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">ZZZ</wsse:Nonce>
<wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2021-03-26T14:35:28+00:00</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
</soap-env:Header>
<soap-env:Body>
<ns0:Renew xmlns:ns0="http://docs.oasis-open.org/wsn/b-2">
<ns0:TerminationTime>2021-03-27T14:35:28Z</ns0:TerminationTime>
</ns0:Renew>
</soap-env:Body>
</soap-env:Envelope>
and the camera response
HTTP/1.1 400 Bad Request
Server: gSOAP/2.8
Content-Type: application/soap+xml; charset=utf-8; action="http://docs.oasis-open.org/wsn/bw-2/SubscriptionManager/RenewRequest"
Content-length: 2012
Connection: close
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:chan="http://schemas.microsoft.com/ws/2005/02/duplex" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:c14n="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" xmlns:wsc="http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:xmime="http://tempuri.org/xmime.xsd" xmlns:xop="http://www.w3.org/2004/08/xop/include" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:wsrfbf="http://docs.oasis-open.org/wsrf/bf-2" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsrfr="http://docs.oasis-open.org/wsrf/r-2" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:tdv="http://www.onvif.org/ver10/events/wsdl" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:tan="http://www.onvif.org/ver20/analytics/wsdl" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl">
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<SOAP-ENV:Code>
<SOAP-ENV:Value>SOAP-ENV:Receiver</SOAP-ENV:Value>
<SOAP-ENV:Subcode>
<SOAP-ENV:Value>ter:InvalidArgVal</SOAP-ENV:Value>
<SOAP-ENV:Subcode>
<SOAP-ENV:Value>ter:ModeError</SOAP-ENV:Value>
</SOAP-ENV:Subcode>
</SOAP-ENV:Subcode>
</SOAP-ENV:Code>
<SOAP-ENV:Reason>
<SOAP-ENV:Text xml:lang="en">
</SOAP-ENV:Reason>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
I have used ONVIF Device Manager v2.2.250 and got the following correct communicating with the same camera (Wanscam K21): Request:
POST /onvif/events_service HTTP/1.1
Content-Type: application/soap+xml; charset=utf-8
Host: 192.168.3.208
Content-Length: 1339
Accept-Encoding: gzip, deflate
Connection: Close
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
<s:Header>
<a:Action s:mustUnderstand="1">http://docs.oasis-open.org/wsn/bw-2/SubscriptionManager/RenewRequest</a:Action>
<a:MessageID>urn:uuid:57c83690-ef2c-4817-aba7-f56b42f3853e</a:MessageID>
<a:ReplyTo>
<a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
</a:ReplyTo>
<Security s:mustUnderstand="1" xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<UsernameToken>
<Username>admin</Username>
<Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">XXX</Password>
<Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">ZZZ</Nonce>
<Created xmlns="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2021-03-26T15:34:58.255Z</Created>
</UsernameToken>
</Security>
<a:To s:mustUnderstand="1">http://192.168.3.208/onvif/events_service</a:To>
</s:Header>
<s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Renew xmlns="http://docs.oasis-open.org/wsn/b-2">
<TerminationTime>PT1M</TerminationTime>
</Renew>
</s:Body>
</s:Envelope>
and response:
HTTP/1.1 200 OK
Server: gSOAP/2.8
Content-Type: application/soap+xml; charset=utf-8
Content-length: 827
Connection: close
<?xml version="1.0" encoding="utf-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:tev="http://www.onvif.org/ver10/events/wsdl">
<wsa5:To SOAP-ENV:mustUnderstand="true">http://192.168.3.208:80/onvif/e</wsa5:To>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<wsnt:RenewResponse>
<wsnt:CurrentTime>2021-03-26T19:34:58Z</wsnt:CurrentTime>
<wsnt:TerminationTime>2021-03-26T19:35:58Z</wsnt:TerminationTime>
</wsnt:RenewResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
It seems SubscriptionManager/RenewRequest is totally broken. I can add other camera XiongMai A5-X20RJ (IPC_XM530_RA50X20) and it works. But I see the same response (400 Bad Request) to RenewRequest
<?xml version='1.0' encoding='utf-8'?>
<soap-env:Envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
<soap-env:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken>
<wsse:Username>admin</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">JQpvpHyTidtpWczlmA5mVT5F+hA=</wsse:Password>
<wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">VHksOlqcRXaawiywHUWIVg==</wsse:Nonce>
<wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2021-03-30T09:46:02+00:00</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
</soap-env:Header>
<soap-env:Body>
<ns0:Renew xmlns:ns0="http://docs.oasis-open.org/wsn/b-2">
<ns0:TerminationTime>2021-03-31T09:46:02Z</ns0:TerminationTime>
</ns0:Renew>
</soap-env:Body>
</soap-env:Envelope>
Response:
<?xml version="1.0" encoding="UTF-8"?>
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:e="http://www.w3.org/2003/05/soap-encoding" xmlns:wsa="http://www.w3.org/2005/08/addressing" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsntw="http://docs.oasis-open.org/wsn/bw-2" xmlns:wsrf-rw="http://docs.oasis-open.org/wsrf/rw-2" xmlns:wsrf-r="http://docs.oasis-open.org/wsrf/r-2" xmlns:wsrf-bf="http://docs.oasis-open.org/wsrf/bf-2" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl" xmlns:wsoap12="http://schemas.xmlsoap.org/wsdl/soap12" xmlns:http="http://schemas.xmlsoap.org/wsdl/http" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:wsadis="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:tns1="http://www.onvif.org/ver10/topics" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:tst="http://www.onvif.org/ver10/storage/wsdl" xmlns:dn="http://www.onvif.org/ver10/network/wsdl" xmlns:tr2="http://www.onvif.org/ver20/media/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:tan="http://www.onvif.org/ver20/analytics/wsdl" xmlns:axt="http://www.onvif.org/ver20/analytics" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:ter="http://www.onvif.org/ver10/error">
<s:Header>
<wsa:Action>http://www.w3.org/2005/08/addressing/soap/fault</wsa:Action>
</s:Header>
<s:Body>
<s:Fault>
<s:Code>
<s:Value>s:Receiver</s:Value>
<s:Subcode>
<s:Value>wsrf-rw:ResourceUnknownFault</s:Value>
</s:Subcode>
</s:Code>
<s:Reason>
<s:Text xml:lang="en">ResourceUnknownFault</s:Text>
</s:Reason>
</s:Fault>
</s:Body>
</s:Envelope>
As the result I don't see any events. I expected the motion alarm/detection at least. But I don't see any sensors at all.
I have partially fixed the issue removing the line https://github.com/home-assistant/core/blob/b58d6a6293ef3f5bd537559516997858abcabcac/homeassistant/components/onvif/event.py#L86
It is necessary to initialize events first before renew anything. After that
- I can add Wanscam K21 and K54 successfully.
- I see a lot of various sensors for XiongMai and Wanscam cameras.
But the problem returns after 1 minute trying to renew subscription. It seems the problem is missing addressing in the Renew request. The Header contains only Security element. Action, To and MessageID are missing in the Header of Renew request. Currently I have no idea how to fix it.
Hope someone of code owners can look at this issue.
This patch must fix the issue https://github.com/mvantellingen/python-zeep/pull/1206
https://github.com/home-assistant/core/pull/48571
what does this mean ? how to fix this now?
https://github.com/home-assistant/core/pull/42651/files
solved this with editing and reverting back the code
@slydiman is this issue now patched in an upcoming release?
Code owner replied no
@slydiman is this issue now patched in an upcoming release?
https://github.com/slydiman/ha_custom_onvif Any comments/improvements are welcome.
@slydiman is this issue now patched in an upcoming release?
https://github.com/slydiman/ha_custom_onvif Any comments/improvements are welcome.
worked for me wow thanks!
@slydiman is this issue now patched in an upcoming release?
https://github.com/slydiman/ha_custom_onvif Any comments/improvements are welcome.
its work for me. Thanks
Hi,
Thanks a lot for your work !
I have now a lot of sensors for a XM530_R80X30-PQ_8M:
I am new with onvif, regarding PTZ, is it something to manage separately to the ONVIF integration ? It would be wonderful to have this automatically configured when add the cam. Like it has been successfully carried out with the app onvifer.
regarding PTZ, is it something to manage separately to the ONVIF integration ?
XM530_R80X30-PQ_8M does not store PTZ presets.
You can add some PTZ presets to the camera using ONVIF Device Manager, then restart HA. Now you can use onvif.ptz service with presets PRESET_0, PRESET_1, etc., but only till reboot the camera.
- service: onvif.ptz
data:
entity_id: camera.cam1_hd
move_mode: GotoPreset
preset: "PRESET_0"
Many thanks,
Will try soon. It remains strange for me that some applications manage this automatically. Will keep you informed.
Hi,
Back on the configuration, I was able to define some presets but I don't know where to add the data :
- service: onvif.ptz data: entity_id: camera.nvt_mainstream move_mode: GotoPreset preset: "PRESET_0"
Are the presets needed to move / zoom ? I wonder not but.
Found, have to add data into the panel.
This example creates buttons to move the camera: https://community.home-assistant.io/t/onvif-camera-platform-how-to-do-ptz/197447/4
Then all good, many thanks for your good job ! and still don't understand why the onvif integration is not taking benefit of your work.
Found, have to add data into the panel.
This example creates buttons to move the camera: https://community.home-assistant.io/t/onvif-camera-platform-how-to-do-ptz/197447/4
Then all good, many thanks for your good job ! and still don't understand why the onvif integration is not taking benefit of your work.
Thanks, its working for me/
This patch must fix the issue mvantellingen/python-zeep#1206
Zeep 4.1.0 has been released yesterday. This patch was merged and reverted immediately with the following comment:
Just reverted it again, needs more thinking (if you always want WSA you could just list it in the plugins)
It means that this issue must be fixed in ONVIF integration which is a part of Home Assistant Core.
Hi,
Thanks for the information regarding the merge of the patch. Does it mean we can remove the personal integration right now ?
Pan & tilt work good. But impossible to get zoom_in & zoom_out to work. Here is what I have in my configuration: - entity: camera.nvt_mainstream tap_action: action: call-service service: onvif.ptz service_data: entity_id: camera.nvt_mainstream zoom: ZOOM_OUT distance: 1 move_mode: ContinuousMove continuous_duration: 1
In the log, there is nothing related to onvif, camera.
Thanks for the information regarding the merge of the patch. Does it mean we can remove the personal integration right now ?
Do you mean my custom ONVIF integration? The patch to Zeep was not applied finally. Nothing changed.
Exactly, I was referring to your custom ONVIF integration. Thank you for your confirmation.
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hello,
I have got a new camera : EYEPLUS_DEV from EYEPLUS Firmware: 5.6.1.1127
I have the main & sub stream but no sensors. How to find / add them to home assistant ?
Many thanks
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
