vertxHttpReqInfo only supports GET (Unable to call HTTP POST or PUT etc)

[OpenSourceTycoon]

Hi There,

I'm able to consume the Kafka Message, but while i trigger the async API call using

Stream<JStreamVertxParallelEoSStreamProcessor.VertxCPResult<String,String>> resultStream = parallelConsumer.vertxHttpReqInfoStream(record -> {
String apiReq = "{id:1}";
// Need to call POST VERB
Map<String, String> params = UniMaps.of("recordKey", "12345", "payload", apiReq );
return new VertxParallelEoSStreamProcessor.RequestInfo("abc.workin.net", 8443, "/api/v1/limit", params);

This is internally calling GET VERB ( not POST VERB ). Can someone help me understand the POST call ?

[OpenSourceTycoon]

@astubbs : Can you help me on this ?

[astubbs]

Hi @OpenSourceTycoon ! Welcome to the project, and thanks for submitting this issue! We appreciate your involvement.

You have indeed uncovered some missing features :) The vertxReqInfo method currently only supports GET requests, as seen on this line.

It shouldn't be hard to add support for the other methods.

However, in the mean time, as a workaround, you can simply use the other methods - either vertxHttpRequest, or vertxWebClient which let you construct any sort of request you want that VertX's WebClient interface supports.

Let me know if that solves your problem.

[OpenSourceTycoon]

@astubbs : can you provide examples for vertxHttpRequest, or vertxWebClient ?

[astubbs]

Sure!

    void webClientExample(VertxParallelStreamProcessor<String, String> parallelConsumer) {
        parallelConsumer.vertxHttpRequest((wc, rec) -> wc.post("some uri"),
                httpResponseFuture -> {
                },
                httpResponseAsyncResult -> {
                }
        );
    }

    void httpRequestExample(VertxParallelStreamProcessor<String, String> parallelConsumer) {
        parallelConsumer.vertxHttpWebClient((wc, rec) -> wc.post("some uri").send(),
                httpResponseFuture -> {
                }
        );
    }

[astubbs]

I've also added a PR to make the API simpler for your use- assuming you aren't interested in using the callback hooks?

https://github.com/confluentinc/parallel-consumer/pull/181/files

[OpenSourceTycoon]

@astubbs PR available for Merge Process. Can you provide the release jars path after merge process. ? Thanks in advance.

[astubbs]

The PR doesn't need to be merged to solve your problem. Just use the "no op" call back functions like in the example above. E.g. :httpResponseFuture -> {}

[OpenSourceTycoon]

Thanks @astubbs. I'm using below code

parallelConsumer.vertxHttpWebClient((wc, rec) -> wc.postAbs("some uri").send(), httpResponseFuture -> { } );

However, i see "Failed to create SSL connection" - Vert.x Vertical Fail : Failed to create SSL connection" issue

[OpenSourceTycoon]

There is a method on the HttpClientOptions, we need to call that method with false. We can use options class in to my code.

[astubbs]

Are you still having an issue? Can you provide. Full failing example perhaps with a broken test?

[OpenSourceTycoon]

@astubbs : Yes. our usecase is

• Pull multiple kafka messages from Confluent Cluster
• Call Parallel API calls to API
• Persist the data

[OpenSourceTycoon]

Please find the code

VertxParallelStreamProcessor<String,String> parallelConsumer;

public void run(){
Consumer<String,String> kafakaConsumer = getKafkaConsumer();

var options = parallelConsumerOptions.<String, String>builder().ordering(KEY).consumer(kafakaConsumer).maxConcurrency(10).build();

this.parallelConsumer = VertxParallelStreamProcessor.createEosStreamProcessor(options);

parallelConsumer.subscrribe(of("test-topic"));

parallelConsumer.vertxHttpWebClient((wc,rec) -> wc.postAbs("https://www.google.com").putHeaders("Content-Type","application/json").sendJson("{}").onSuccess(response -> { log.info("Response""+response.bodyAsString())}
.onFailure(err -> log.info("Failure"); }), httpResponseFuture -> {
});

}

[astubbs]

What's the issue?

[OpenSourceTycoon]

2021-11-25 11:47:44.084 [vert.x-eventloop-thread-1] INFO c.j.c.r.u.a.c.ParallelConsumer - Failure --Failed to create SSL connection 2021-11-25 11:47:44.084 [vert.x-eventloop-thread-1] ERROR i.c.p.v.VertxParallelEoSStreamProcessor - Vert.x Vertical fail: Failed to create SSL connection

[OpenSourceTycoon]

our port is 8443 not 443. vert.x function validating the SSL Certificate and not able to see the CIPHER. May be we can set - HttpClinetOptions.setTrustAll(true); Not sure ..how to map with webclient object.

[OpenSourceTycoon]

I'm referring this example

[OpenSourceTycoon]

Hi Antony,

I updated the thread - https://github.com/confluentinc/parallel-consumer/issues/180

On Thu, Nov 25, 2021 at 9:00 PM Antony Stubbs @.***> wrote:

What's the issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/confluentinc/parallel-consumer/issues/180#issuecomment-979304855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABF5VWLSDMBUPQ3JVTFKEDUNZJCHANCNFSM5ITPSBBQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Krishna Rao Veeramachaneni

[ozturkberkay]

@OpenSourceTycoon Were you able to solve this issue?