Sharing threads between HttpClient instances

[dennis-yemelyanov]

I have a requirement to allow different proxy settings for each host (based on standard ProxySelector in Java, which takes in a URI to determine proxy settings for a given call). To do that it seems like I have to create a different HttpClient instance per host.

The issue is that when creating and starting multiple HttpClient instances, the number of threads it creates also grows proportionally. For example, if I create 3 instances, there will be about 30 threads instead of 10 threads for one instance. But I know that 10 threads would be enough since there would be the same amount of work as if there was only one HttpClient instance, it will just be spread across multiple instances.

So essentially, is there any way I can make Example 1 below use the same number of threads as Example 2?

Or is it possible to meet the original requirement (different proxy per host) without creating multiple HttpClient instances? It just feels a bit sub-optimal to create more threads just to be able to specify different proxy settings.

Example 1:

QueuedThreadPool pool = new QueuedThreadPool();           
ScheduledExecutorScheduler scheduler = new ScheduledExecutorScheduler("default-scheduler", false);

for (int i = 0; i < 3; i++) {
    HttpClient httpClient = new HttpClient();
    httpClient.setExecutor(pool);
    httpClient.setScheduler(scheduler);
    httpClient.start();

    System.out.println(httpClient.newRequest("https://example.com").send().getStatus());
}

Example 2:

QueuedThreadPool pool = new QueuedThreadPool();           
ScheduledExecutorScheduler scheduler = new ScheduledExecutorScheduler("default-scheduler", false);

HttpClient httpClient = new HttpClient();
httpClient.setExecutor(pool);
httpClient.setScheduler(scheduler);
httpClient.start();

for (int i = 0; i < 3; i++) {
    System.out.println(httpClient.newRequest("https://example.com").send().getStatus());
}

[sbordet]

is it possible to meet the original requirement (different proxy per host)

Yes, see the documentation: https://eclipse.dev/jetty/documentation/jetty-12/programming-guide/index.html#pg-client-http-proxy

[janbartel]

Closing issue as answered.