Enabling compression and chunking in MarkLogic 10 in Java Client API

[llinggit]

Please refer to https://bugtrack.marklogic.com/57132 "It has been decided that for ML10 we will not automatically turn on HTTP chunking/compression in ML10 based on request headers, see; https://bugtrack.marklogic.com/57123 However compression will improve user experience and chunking allow customers to detect errors while streaming.

A lot of customers are using CURL, Mulesoft, TIBCO, to download (and transport) datasets from ML, so we need to add an option to 'high volume' endpoints (e.g. /v1/rows) to force enable gzip/chunking. Those endpoint(s) should call: xdmp:set-response-chunked-encoding(fn:true()) and/or xdmp:set-response-compression(fn:true()) . This code is already in trunk/b10 so there is no need to wait on this feature.

I would recommend to have an option which also work outside the ML JavaClient API (e.g. an URL parameter), because most customers (in my experience) use standard tools for this (such as CURL, TIBCO etc) to do this plumbing. Others use indirectly the ML JavaClient API (e.g. NIFI), but then this requires a new NIFI connector "compiled" to a newer Java Client which is more difficult than adding a URL parameter. Typically those tools already support chunked/compression or they have options to support this, e.g. by feeding the input through a unzip workflow. "

So we can address your issue, please include the following:

Version of MarkLogic Java Client API

See Readme.txt

Version of MarkLogic Server

See admin gui on port 8001 or run xdmp:version() in Query Console - port 8000)

Java version

Run java -version

OS and version

For MAC, run sw_vers.
For Windows, run systeminfo | findstr /B /C:"OS Name" /C:"OS Version"
For Linux, run cat /etc/os-release and uname -r

Input: Some code to illustrate the problem, preferably in a state that can be independently reproduced on our end

Actual output: What did you observe? What errors did you see? Can you attach the logs? (Java logs, MarkLogic logs)

Expected output: What specifically did you expect to happen?

Alternatives: What else have you tried, actual/expected?