Streaming HTTP request with Webclient blocks

[horak90]

Version

I am using vert.x version 4.5.13

Context

I am attempting to stream a large HTTP request between my Vert.x services. For this purpose, I am using io.vertx.reactivex.ext.web.client.WebClient (created with HttpEndpoint.rxGetWebClient(serviceDiscovery, rec ->rec.getName().equals(serviceName), webClientOptions)), which takes a Flowable as input for the rxSendStream method.

The Flowable in question is based on a ReplaySubject populated with buffers generated by another method.

However, I have encountered an issue where the other service does not receive all the buffers. Specifically:

The HTTP request appears to emit a certain number of buffers and then blocks, leaving the request incomplete.

On the consumer side, fewer buffers are received than expected, and the process becomes stuck without completing the request.

I have tested the behavior with Flowable directly (without using ReplaySubject), and the result appears to be the same.

Do you have a reproducer?

https://github.com/horak90/webclient_reproducer

[vietj]

can you explain how to use the reproducer ?

[tsegismont]

@vietj I was looking at this

You can add this to the main verticle class:

  public static void main(String[] args) {
    Vertx vertx = Vertx.vertx();
    vertx.rxDeployVerticle(new MainVerticle()).blockingGet();
  }

It seems it works fine with 4.5.1 and fails with 4.5.2+

[tsegismont]

This appears to be a regression due to https://github.com/vert-x3/vertx-rx/pull/304

When updating the reproducer to use 4.5.2 for the whole stack and 4.5.1 for vertx-rx-java2 , it works.

[tsegismont]

@vietj reading vert-x3/vertx-rx#304, it seems the original issue was solved in https://github.com/eclipse-vertx/vert.x/issues/4982

So I'm not sure why https://github.com/vert-x3/vertx-rx/pull/304 was merged. Can you check?

[vietj]

it was fixing a bug :-)

[tsegismont]

it was fixing a bug :-)

But which one? In the same PR, there's a comment that says the original issue was solved in https://github.com/eclipse-vertx/vert.x/issues/4982

[jhaeyaert]

Hi,

We've noticed the same issue on Gravitee when making an HTTP request (HTTP 1.1 with chunked transfer or HTTP2 stream). The issue seems to be occurring because of https://github.com/vert-x3/vertx-rx/blob/4.5.2/rx-java3-gen/src/main/java/io/vertx/rxjava3/impl/ReadStreamSubscriber.java#L120-L122

Previously, the checkStatus() method was systematically called when calling the onNext(item) ensuring that, in the end, a call to subscription.request(request) was made.

Since the fix in vertx-rx 4.5.2, the upstream isn't requested anymore leading to a request stuck waiting on both the downstream and upstream sides.

Since this PR seems to fix the issue, can the vertx-rx PR be reverted?

Thanks

[tsegismont]

@jhaeyaert thanks for your feedback.

I know @vietj is busy on other priorities right now but for sure we'll answer that question before 4.5.14 goes out

[vietj]

before reverting anything in the bind I would like to get a reproducer that passes when the fix is reverted