Support for gzip encoding

[martonmiklos]

I need to communicate with a webservice which requires gzip compression of the transmitted SOAP envelope.

I have a .NET+C# based example configured with source code available, so I have managed to figure out what needs to be compressed (whole envelope) and what content type (application/x-gzip) shall I send.

In the WSDL the GZIP compression is enabled by the following tag (so even kdwsdl2cpp could support it):

<wsdl:definitions ...... >
  <wsp:Policy wsu:Id="CustomBinding_ITransactions_policy">
    <wsp:ExactlyOne>
      <wsp:All>
        **<gzip:GZipEncoding xmlns:gzip="http://schemas.microsoft.com/ws/06/2004/mspolicy/netgzip1"/>**

I will patch the KDSOAP to support his feature, and if I do so I would like contribute this change back to the upstream.

My questions would be the following:

• Is KDAB open for this contribution?
• Since Qt itself is not includes functions for gzip compression it would be straightforward to use external library for compression. I have found KArchive which supports gzip and it is a Tier 1 KDE library which means does not require any external dependecy other than Qt. Do you see any objections of including this as a submodule?

[dfaure-kdab]

Hah, this is interesting. Given that I'm both the maintainer of KArchive and of KDSoap, I can hardly object to KDSoap using KArchive :-)

On the other hand, KArchive's main added value is archive-like formats (TAR, ZIP etc.), while simply gzipping can be done rather easily with zlib directly. In fact, KArchive's KCompressionDevice makes the API a bit nicer by behaving like a QIODevice, but that API isn't made for network-based data (push rather than pull, i.e. we can't just ask for more data from the underlying device and get it right away). I remember investigating that and arriving to that conclusion, see KFilterTest::test_pushData in the karchive autotests.

There is actually code in KIO that implements on-demand deflating on top of KFilterBase, but we can't use that in KDSoap (due to the commercial license), it needs to be written from scratch.

BTW does this end up using the standard HTTP "Accept-Encoding: [gzip|deflate|compress]" feature or is this a Microsoft extension done differently? I guess the latter, otherwise QNetworkAccessManager would do all this automatically for us, right?

Anyhow, I am open to the contribution, but I suggest using zlib directly.

[martonmiklos]

Given that I'm both the maintainer of KArchive and of KDSoap, I can hardly object to KDSoap using KArchive :-)

Yeah I have just seen that yesterday, and I found it funny.

BTW does this end up using the standard HTTP "Accept-Encoding: [gzip|deflate|compress]" feature or is this a Microsoft extension done differently? I guess the latter, otherwise QNetworkAccessManager would do all this automatically for us, right?

I have to double check the headers sent by the server, and check how the KDSOAP handles the Accept-Encoding headers. Without configuring anything in KDSOAP my requests got refused because of the mime type sent by the KDSOAP.

but I suggest using zlib directly.

Do you suggest to link to the zlib library, or I should add it as a submodule and include the sources directly? The linking option is more elegant, however we could ran into issues on Windows.

[martonmiklos]

The server sends the following headers (in the response to sending properly gzipped payload):

"Content-Length" "504"
"Content-Type" "application/x-gzip"
"Server" "Microsoft-IIS/7.5"
"X-Powered-By" "ASP.NET"
"Date" "Mon, 16 Dec 2019 21:21:23 GMT"

I have checked the QHttpNetworkReply code and it expects the "content-encoding" header to be set to "gzip" or "deflate", so this is something different: https://github.com/qt/qtbase/blob/dev/src/network/access/qhttpnetworkreply.cpp#L388

I have added zlib as a submodule, and mainly get it rolling, you can see my results here: https://github.com/martonmiklos/KDSoap/tree/gzip_encoding

I will work on adding some tests, and I would like to figure out how to proceed with #198 first.

[dfaure-kdab]

Ah indeed. This is HTTP sending a gzipped document, not HTTP itself using gzip encoding for a document. No idea why they do this, but OK, no way to handle this in QNetworkAccessManager (if you're downloading a .gz file, it should arrive compressed, not uncompressed).

Thanks, I'll look at #198 now.