Unable to Stream Chunked Response

[zirota]

Hello,

I am using spring cloud gateway mvc: 4.1.5 When making requests that expects a streaming chunked response - where each chunk is returned to the api caller as soon as a chunk is ready. The gateway returns the response in different batches, each with multiple chunks at the same time.

I found a similar issue raised here: https://github.com/spring-cloud/spring-cloud-gateway/issues/2868

I don't think its that streaming is not supported as curl is return the response from the gateway in batches - instead of the entire response being returned all at once. It could be that the gateway is somehow caching the response chunks. Is there configuration that can be done on the gateway to resolve it?

Sample I have tried a variety of different approaches. Including the examples in the linked issue.

My original approach using just handler functions:

defaultRoute = route("some-route")
                .route(path(this.targetApiPrefix), https(uri))
                .build();

Attempt to use of StreamingResponseBody with ProxyExchange

@PostMapping("/api/v1/chat")
public ResponseEntity<?> proxyChatApi(
        @RequestHeader(HttpHeaders.AUTHORIZATION) String authorizationHeader,
        @RequestBody Map body,
        ProxyExchange<?> proxyExchange) throws Exception {

    proxyExchange.header(HttpHeaders.AUTHORIZATION, authorizationHeader);
    proxyExchange.body(body);
    var response = proxyExchange.uri(this.uri).post();

    StreamingResponseBody stream = outputStream -> {
        StreamUtils.copy(response.getBody().getContentAsByteArray(), outputStream);
    };
    return new ResponseEntity(stream, response.getStatusCode());
}

Attempt to return a Flux<?> with ProxyExchange - I am not very familiar with the reactive way so pardon me if this seems incorrect:

@PostMapping("/api/v1/chat")
public Flux<?> proxyChatApi(
        @RequestHeader(HttpHeaders.AUTHORIZATION) String authorizationHeader,
        @RequestBody Map body,
        ProxyExchange<?> proxyExchange) throws Exception {
    var uri = new URI(this.uri);
    proxyExchange.header(HttpHeaders.AUTHORIZATION, authorizationHeader);
    proxyExchange.body(body);
    var response = proxyExchange.uri(uri.toString()).post();
    return Flux.from(response);
}