undeterministic behavior of parse_xmlelements

[Kunzol]

Zeep Version 2.4.0 Python: 3.5.2 OS: Ubuntu 16.04.3 LTS (xenial)

I have a very strange problem. I mean "strange" because the output is somehow not deterministic. Which means, sometimes it works as expected and a second later not.

This is the example code (from "Reporting bugs):

import pretend  # pip install pretend

from zeep import Client
from zeep.transports import Transport

# Replace YOUR-WSDL and OPERATION_NAME with the wsdl url
# and the method name you are calling. The response
# needs to be set in the content=""" """ var.

client = Client('resource.wsdl')
response = pretend.stub(
    status_code=200,
    headers={},
    content="""<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Body><ns2:saveResponse xmlns:ns2="http://endpoint.webservice.component.bfabric.org/"><return><resource><errorreport>You have not the permission to perform this update.</errorreport></resource></return></ns2:saveResponse></S:Body></S:Envelope>
    """)

operation = client.service._binding._operations['save']
result = client.service._binding.process_reply(
    client, operation, response)

print(result)

WSDL and XSD can be found in the gist: https://gist.github.com/Kunzol/cc1b8156a66505d835b5066e4aab9e3e.js

This is the output I expect:

$ python3 parse.py 
{
    'entitiesonpage': None,
    'errorreport': None,
    'numberofpages': None,
    'page': None,
    'access': [],
    'annotation': [],
    'application': [],
    'attachment': [],
    'comment': [],
    'dataset': [],
    'executable': [],
    'externaljob': [],
    'importresource': [],
    'link': [],
    'mail': [],
    'orderitem': [],
    'order': [],
    'parameter': [],
    'projectcharge': [],
    'project': [],
    'resource': [
        {
            'created': None,
            'createdby': None,
            'modified': None,
            'modifiedby': None,
            'status': None,
            'deletionreport': None,
            'errorreport': 'You have not the permission to perform this update.',
            'comment': [],
            'name': None,
            'filechecksum': None,
            'relativepath': None,
            'size': None,
            'description': None,
            'expirationdate': None,
            'sample': None,
            'project': None,
            'report': None,
            'storage': None,
            'uris': [],
            'url': None,
            'inputresource': None,
            'junk': None,
            'junkcomment': None,
            'succeedingworkunit': [],
            'workunit': None,
            'id': None
        }
    ],
    'role': [],
    'sample': [],
    'storage': [],
    'user': [],
    'workunit': []
}

and rerun it a few seconds later I get the error:

$ python3 parse.py 
Traceback (most recent call last):
  File "parse.py", line 19, in <module>
    client, operation, response)
  File "XXX/zeep/wsdl/bindings/soap.py", line 185, in process_reply
    result = operation.process_reply(doc)
  File "XXX/zeep/wsdl/bindings/soap.py", line 366, in process_reply
    return self.output.deserialize(envelope)
  File "XXX/zeep/wsdl/messages/soap.py", line 90, in deserialize
    body_result = self._deserialize_body(body)
  File "XXX/zeep/wsdl/messages/soap.py", line 504, in _deserialize_body
    result = self.body.parse(response_element, self.wsdl.types)
  File "XXX/zeep/xsd/elements/element.py", line 114, in parse
    schema_type=self.type)
  File "XXX/zeep/xsd/types/complex.py", line 169, in parse_xmlelement
    elements, schema, name, context=context)
  File "XXX/zeep/xsd/elements/indicators.py", line 600, in parse_xmlelements
    xmlelements, schema, name, context=context)
  File "XXX/zeep/xsd/elements/element.py", line 160, in parse_xmlelements
    xmlelement, schema, allow_none=True, context=context)
  File "XXX/zeep/xsd/elements/element.py", line 114, in parse
    schema_type=self.type)
  File "XXX/zeep/xsd/types/complex.py", line 169, in parse_xmlelement
    elements, schema, name, context=context)
  File "XXX/zeep-test/zeep/xsd/elements/indicators.py", line 600, in parse_xmlelements
    xmlelements, schema, name, context=context)
  File "XXX/zeep/xsd/elements/element.py", line 160, in parse_xmlelements
    xmlelement, schema, allow_none=True, context=context)
  File "XXX/zeep/xsd/elements/element.py", line 114, in parse
    schema_type=self.type)
  File "XXX/zeep/xsd/types/complex.py", line 178, in parse_xmlelement
    raise XMLParseError("Unexpected element %r" % elements[0].tag)
zeep.exceptions.XMLParseError: Unexpected element 'errorreport'

The error and the correct result can not be predicted. Sometimes I have to run it 10 times, before getting the correct result and sometimes it runs 10 times before getting the error.

This is just one example of my endpoints. There is similar behavior with other endpoints, but this is the "worst" (means more errors than correct results).

With "strict=False" the error does not come up, but the result is different than expected. The _raw_element is filled in this case.

Could this be a problem with the underlaying XML/XSD parser ? Or a WSDL/XSD size problem?

Any ideas ?

Thanks for your help in advance.

[mvantellingen]

I had some time to look at zeep issues again, but looking at this issue i ran into the problem that the wsdl is incomplete. So I can't reproduce it :-(

Can you look into that?

[Kunzol]

I tried hard to give you all the scripts and data needed. I remember, that I could reproduce the problem with the scripts and data I added to this issue.

The first thing I found was that the link to the gist seems broken ... resource.wsdl Additionally I just saw now that it is not complete ... I am sorry for that and I will update it asap.

As discussed in #658 it needs quit large xsd files.

[Kunzol]

The last line (closing schema) in the wsdl was missing.

I am in the same position as @acidjunk. Just a user of zeep, without deep knowledge of the underlying technology. But I try hard to give you all information needed. If a day would have more than 24 hours, I would learn all the things needed to solve the problem :smile:

Just to give you some background, at the moment I do all the Soap things in bash with xmlstarlet, but I want to switch to python, because nobody can use my "cryptic" bash scripts. Unfortunately in my concrete Soap usage, zeep is doing to strict checking of the xsl/wsdl. This is a problem due to a design error of the server side (5 years ago), which was also partly my fault, because of my lack of knowledge about xsl/wsdl :angry:

[acidjunk]

The funny thing is dat in my MR: I pinpoint the regression -> that seems to be caused by fixing some issue from a user that didn't provide an example nor use-case. That resulted in a fix (which caused the regression). That user never replied or acknowledged the fix. A unit test was also added (which seems flawed).

There was a serious amount of time invested in providing test, WSDL's and a MR; but somehow the user that didn't provide test/examples/replies: got handled. It just doesn't feel right.