micro-integrator icon indicating copy to clipboard operation
micro-integrator copied to clipboard
verified publisher icon wso2

HTTP Connector Fails to Handle Brotli-Encoded Responses

Open KalinduGandara opened this issue 10 months ago • 3 comments

Description

When making an HTTP request using the WSO2 Micro Integrator (MI) HTTP Connector, if the response is Brotli-encoded (Content-Encoding: br), the request fails with an unclear error message. This makes it difficult to diagnose and resolve the issue.

related issue - https://github.com/wso2/product-micro-integrator/issues/3028

[2025-02-18 14:46:57,658] ERROR {JsonStreamBuilder} - Error occurred while processing document for application/json java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.wso2.micro.integrator.core.json.JsonStreamBuilder.processDocument(JsonStreamBuilder.java:69)
	at org.apache.synapse.transport.passthru.util.DeferredMessageBuilder.getDocument(DeferredMessageBuilder.java:153)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:179)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:132)
	at org.apache.synapse.transport.util.PassThroughMessageHandler.buildMessage(PassThroughMessageHandler.java:103)
	at org.apache.synapse.util.MediatorEnrichUtil.buildMessage(MediatorEnrichUtil.java:248)
	at org.apache.synapse.mediators.template.InvokeMediator.doEnrich(InvokeMediator.java:533)
	at org.apache.synapse.mediators.template.InvokeMediator.processConnectorResponse(InvokeMediator.java:506)
	at org.apache.synapse.mediators.template.InvokeMediator.postMediate(InvokeMediator.java:387)
	at org.apache.synapse.mediators.template.InvokeMediator.mediate(InvokeMediator.java:257)
	at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:249)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.mediateFromContinuationStateStack(Axis2SynapseEnvironment.java:857)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:330)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:678)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:213)
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
	at org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:339)
	at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.axis2.AxisFault: #Can not parse stream. MessageID: urn:uuid:6a9764a9-c599-4f40-a579-af6ff156a912. Error>>> #getNewJsonPayload. Could not save JSON payload. Invalid input stream found. Payload is not a JSON string.
	at org.apache.synapse.commons.json.JsonUtil.getNewJsonPayload(JsonUtil.java:904)
	at org.apache.synapse.commons.json.JsonStreamBuilder.processDocument(JsonStreamBuilder.java:43)
	... 25 more
Caused by: org.apache.axis2.AxisFault: #getNewJsonPayload. Could not save JSON payload. Invalid input stream found. Payload is not a JSON string.
	at org.apache.synapse.commons.json.JsonUtil.getNewJsonPayload(JsonUtil.java:888)
	... 26 more

[2025-02-18 14:46:57,658] ERROR {DeferredMessageBuilder} - Error building message org.apache.axis2.AxisFault
	at org.wso2.micro.integrator.core.json.JsonStreamBuilder.processDocument(JsonStreamBuilder.java:73)
	at org.apache.synapse.transport.passthru.util.DeferredMessageBuilder.getDocument(DeferredMessageBuilder.java:153)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:179)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:132)
	at org.apache.synapse.transport.util.PassThroughMessageHandler.buildMessage(PassThroughMessageHandler.java:103)
	at org.apache.synapse.util.MediatorEnrichUtil.buildMessage(MediatorEnrichUtil.java:248)
	at org.apache.synapse.mediators.template.InvokeMediator.doEnrich(InvokeMediator.java:533)
	at org.apache.synapse.mediators.template.InvokeMediator.processConnectorResponse(InvokeMediator.java:506)
	at org.apache.synapse.mediators.template.InvokeMediator.postMediate(InvokeMediator.java:387)
	at org.apache.synapse.mediators.template.InvokeMediator.mediate(InvokeMediator.java:257)
	at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:249)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.mediateFromContinuationStateStack(Axis2SynapseEnvironment.java:857)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:330)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:678)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:213)
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
	at org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:339)
	at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[2025-02-18 14:46:57,658] ERROR {RelayUtils} - Error while building Passthrough stream org.apache.axis2.AxisFault
	at org.wso2.micro.integrator.core.json.JsonStreamBuilder.processDocument(JsonStreamBuilder.java:73)
	at org.apache.synapse.transport.passthru.util.DeferredMessageBuilder.getDocument(DeferredMessageBuilder.java:153)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:179)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:132)
	at org.apache.synapse.transport.util.PassThroughMessageHandler.buildMessage(PassThroughMessageHandler.java:103)
	at org.apache.synapse.util.MediatorEnrichUtil.buildMessage(MediatorEnrichUtil.java:248)
	at org.apache.synapse.mediators.template.InvokeMediator.doEnrich(InvokeMediator.java:533)
	at org.apache.synapse.mediators.template.InvokeMediator.processConnectorResponse(InvokeMediator.java:506)
	at org.apache.synapse.mediators.template.InvokeMediator.postMediate(InvokeMediator.java:387)
	at org.apache.synapse.mediators.template.InvokeMediator.mediate(InvokeMediator.java:257)
	at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:249)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.mediateFromContinuationStateStack(Axis2SynapseEnvironment.java:857)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:330)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:678)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:213)
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
	at org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:339)
	at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[2025-02-18 14:46:57,658] ERROR {MediatorEnrichUtil} - Error while building message. Error while building Passthrough stream org.apache.axis2.AxisFault: Error while building Passthrough stream
	at org.apache.synapse.transport.passthru.util.RelayUtils.handleException(RelayUtils.java:467)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:230)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:132)
	at org.apache.synapse.transport.util.PassThroughMessageHandler.buildMessage(PassThroughMessageHandler.java:103)
	at org.apache.synapse.util.MediatorEnrichUtil.buildMessage(MediatorEnrichUtil.java:248)
	at org.apache.synapse.mediators.template.InvokeMediator.doEnrich(InvokeMediator.java:533)
	at org.apache.synapse.mediators.template.InvokeMediator.processConnectorResponse(InvokeMediator.java:506)
	at org.apache.synapse.mediators.template.InvokeMediator.postMediate(InvokeMediator.java:387)
	at org.apache.synapse.mediators.template.InvokeMediator.mediate(InvokeMediator.java:257)
	at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:249)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.mediateFromContinuationStateStack(Axis2SynapseEnvironment.java:857)
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:330)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:678)
	at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:213)
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
	at org.apache.synapse.transport.passthru.ClientWorker.run(ClientWorker.java:339)
	at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.axis2.AxisFault
	at org.wso2.micro.integrator.core.json.JsonStreamBuilder.processDocument(JsonStreamBuilder.java:73)
	at org.apache.synapse.transport.passthru.util.DeferredMessageBuilder.getDocument(DeferredMessageBuilder.java:153)
	at org.apache.synapse.transport.passthru.util.RelayUtils.buildMessage(RelayUtils.java:179)
	... 18 more

Steps to Reproduce

  1. Configure an HTTP request in WSO2 MI using the HTTP Connector.
  2. Send a GET request to a server that responds with Brotli encoding.
  3. Observe that the response fails with an unclear error message.

Version

4.4.0 - RC2

Environment Details (with versions)

No response

KalinduGandara avatar Feb 18 '25 09:02 KalinduGandara

Hi! I’d like to work on this issue. Could you please assign this issue to me?

Lakshan-Chamoditha-Perera avatar Sep 26 '25 08:09 Lakshan-Chamoditha-Perera

Hi Ojith I am happy to work on this issue. Could you please assign this issue to me?

OjithUddeeptha avatar Sep 29 '25 14:09 OjithUddeeptha

Hey Kalindu, I'd be more than happy to work on this issue. Can you assign this to me?

AVDiv avatar Sep 30 '25 05:09 AVDiv