Garbage in nested datatypes

[elsampsa]

Intro

This example case comes from the infamous onvif world (for which there still isn't any working soap client)

It's a bit too complicated to be forged into the standard debug template so feel free to close this one immediate (this repo seems unmaintained anyway). Anyway, I felt like documenting this.

Any comments / advices appreciated. Did I get this right?

The Bug

So, it seems that Zeep breaks response objects for nested datatypes.

First, consider the following nested data structure:

name[type]

Extension[CapabilitiesExtension]
    ...
    DeviceIO[DeviceIOCapabilities]
        ...
        AudioSources[int]

that is used by

https://www.onvif.org/ver10/device/wsdl/devicemgmt.wsdl

in GetCapabilities call. The variable "Extension" in the nested return value is of the type CapabilitiesExtension and is defined in onvif.xsd like this:

<xs:complexType name="CapabilitiesExtension">
		<xs:sequence>
			<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
			<xs:element name="DeviceIO" type="tt:DeviceIOCapabilities" minOccurs="0"/>
			<xs:element name="Display" type="tt:DisplayCapabilities" minOccurs="0"/>
			<xs:element name="Recording" type="tt:RecordingCapabilities" minOccurs="0"/>
			<xs:element name="Search" type="tt:SearchCapabilities" minOccurs="0"/>
			<xs:element name="Replay" type="tt:ReplayCapabilities" minOccurs="0"/>
			<xs:element name="Receiver" type="tt:ReceiverCapabilities" minOccurs="0"/>
			<xs:element name="AnalyticsDevice" type="tt:AnalyticsDeviceCapabilities" minOccurs="0"/>
			<xs:element name="Extensions" type="tt:CapabilitiesExtension2" minOccurs="0"/>
		</xs:sequence>
	</xs:complexType>
...
...
<xs:complexType name="DeviceIOCapabilities">
		<xs:sequence>
			<xs:element name="XAddr" type="xs:anyURI"/>
			<xs:element name="VideoSources" type="xs:int"/>
			<xs:element name="VideoOutputs" type="xs:int"/>
			<xs:element name="AudioSources" type="xs:int"/>
			<xs:element name="AudioOutputs" type="xs:int"/>
			<xs:element name="RelayOutputs" type="xs:int"/>
			<xs:any namespace="##any" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
		</xs:sequence>
		<xs:anyAttribute processContents="lax"/>
	</xs:complexType>
...
...

Another example:

name[type]

Media[MediaCapabilities]
    ...
    StreamingCapabilities[RealTimeStreamingCapabilities]
        ...
        RTSPMulticast[boolean]

The MediaCapabilities type is defined in onvif.xsd like this:

<xs:complexType name="MediaCapabilities">
		<xs:sequence>
			<xs:element name="XAddr" type="xs:anyURI">
				<xs:annotation>
					<xs:documentation>Media service URI.</xs:documentation>
				</xs:annotation>
			</xs:element>
			<xs:element name="StreamingCapabilities" type="tt:RealTimeStreamingCapabilities">
				<xs:annotation>
					<xs:documentation>Streaming capabilities.</xs:documentation>
				</xs:annotation>
			</xs:element>
			<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>
			<xs:element name="Extension" type="tt:MediaCapabilitiesExtension" minOccurs="0"/>
		</xs:sequence>
		<xs:anyAttribute processContents="lax"/>
</xs:complexType>
...
...
<xs:complexType name="RealTimeStreamingCapabilities">
		<xs:sequence>
			<xs:element name="RTPMulticast" type="xs:boolean" minOccurs="0">
				<xs:annotation>
					<xs:documentation>Indicates whether or not RTP multicast is supported.</xs:documentation>
				</xs:annotation>
			</xs:element>
			<xs:element name="RTP_TCP" type="xs:boolean" minOccurs="0">
				<xs:annotation>
					<xs:documentation>Indicates whether or not RTP over TCP is supported.</xs:documentation>
				</xs:annotation>
			</xs:element>
			<xs:element name="RTP_RTSP_TCP" type="xs:boolean" minOccurs="0">
				<xs:annotation>
					<xs:documentation>Indicates whether or not RTP/RTSP/TCP is supported.</xs:documentation>
				</xs:annotation>
			</xs:element>
			<xs:element name="Extension" type="tt:RealTimeStreamingCapabilitiesExtension" minOccurs="0"/>
		</xs:sequence>
		<xs:anyAttribute processContents="lax"/>
	</xs:complexType>

That's also used by

https://www.onvif.org/ver10/device/wsdl/devicemgmt.wsdl

When calling the GetCapabilities method, we get in return a GetCapabilitiesResponse object, which Zeep returns like this:

...
...

'Extension': {              # type: CapabilitiesExtension 
        '_value_1': [
            <Element {http://www.onvifext.com/onvif/ext/ver10/schema}hikCapabilities at 0x7fde69390d08>,   # unresolved variables?
            <Element {http://www.onvif.org/ver10/schema}DeviceIO at 0x7fde69390e08>
        ],
    'DeviceIO': None,       # These should not be here!
    'Display': None,
    'Recording': None,
    'Search': None,
    'Replay': None,
    'Receiver': None,
    'AnalyticsDevice': None,
    'Extensions': None
},
...
...
'Media': {                  # type: MediaCapabilities
    'XAddr': 'http://192.168.0.157/onvif/Media',
    'StreamingCapabilities': {
        'RTPMulticast': True,
        'RTP_TCP': True,
        'RTP_RTSP_TCP': True,
        'Extension': None,
        '_attr_1': None
    },
    '_value_1': [
        <Element {http://www.onvif.org/ver10/schema}Extension at 0x7f881fa35208>
    ],
    'Extension': None,
    '_attr_1': None
},
...
...

So for the "Extension" variable, the query went sour .. Instead, we have, under the key _value_1 some xml elements. One of them is:

<Element {http://www.onvif.org/ver10/schema}DeviceIO at 0x7fde69390e08>

It seems to be an unresolved variable of type "DeviceIO" from namespace "http://www.onvif.org/ver10/schema". But DeviceIO is not a type, instead its a variable name..! That should be a variable of the type "DeviceIOCapabilities" from namespace "http://www.onvif.org/ver10/schema"

There seems to be some hard-core namespace/variable type/name confusion going on here.

On the other hand, the "Media" variable is for most part, just fine.

Wiresharking the SOAP response from which the response objects were constructed gives:

<env:Body>
    <tds:GetCapabilitiesResponse>
        <tds:Capabilities>
            <tt:Analytics>
            ...
            ...
            </tt:Analytics>
            ...
            ...
            <tt:Media>
                <tt:XAddr>
                    http://192.168.0.157/onvif/Media
                </tt:XAddr>
                <tt:StreamingCapabilities>  # variable "StreamingCapabilities" .. Zeep gets the type "RealTimeStreamingCapabilities" right ok (see above)
                    <tt:RTPMulticast>
                        true
                    </tt:RTPMulticast>
                    <tt:RTP_TCP>
                        true
                    </tt:RTP_TCP>
                    <tt:RTP_RTSP_TCP>
                        true
                    </tt:RTP_RTSP_TCP>
                </tt:StreamingCapabilities>
                <tt:Extension>
                    <tt:ProfileCapabilities>
                        <tt:MaximumNumberOfProfiles>
                            10
                        </tt:MaximumNumberOfProfiles>
                    </tt:ProfileCapabilities>
                </tt:Extension>
            </tt:Media>
                ...
                ...
            <tt:Extension>
                ...
                ...
                <tt:DeviceIO>                # variable "DeviceIO" .. Zeep never gets the type "DeviceIOCapabilities" right (see above)
                    <tt:XAddr>
                        http://192.168.0.157/onvif/DeviceIO
                    </tt:XAddr>
                    <tt:VideoSources>
                        1
                    </tt:VideoSources>
                    <tt:VideoOutputs>
                        0
                    </tt:VideoOutputs>
                    <tt:AudioSources>
                        1
                    </tt:AudioSources>
                    <tt:AudioOutputs>
                        1
                    </tt:AudioOutputs>
                    <tt:RelayOutputs>
                        0
                    </tt:RelayOutputs>
                </tt:DeviceIO>
            </tt:Extension>
        </tds:Capabilities>
    </tds:GetCapabilitiesResponse>
</env:Body>

[EmpireofKings]

Bump