Recommendation for when to use a client connection pool

[AntonOyung]

Is there a rough limit (or guideline) for when it's recommended we maintain a pool of ClickHouseClients instead of generating a new client per request? (Whether it's a limit of total open clients at any given time, or frequency of writes, or volume of writes, etc)?

I'm developing an application with frequent writes of high volume on multiple threads and it seems like it wouldn't be performant to generate new clients per request

Originally posted by @AntonOyung in https://github.com/ClickHouse/clickhouse-java/issues/786#issuecomment-1908937224

[ladislavmacoun]

I'm uncertain about the pooling strategy, but if you're utilizing an HTTP client, configuring the SOCKET_KEEPALIVE and SOCKET_REUSEADDR options can help reduce the total number of open connections. This approach has proven effective for us at scale.

[AntonOyung]

Thanks for the response! So it sounds like we should be reusing clients instead of generating new ones per request/write.

If I have for example have 30 JVMs each with 30 threads running against my cluster, is it recommended we maintain 1 ClickHouseClient per thread? From a cursory look of the code it seems like each client maintains its own pool of connections, so that would be 30 * 30 * (X # of connections established by the client) total connections.

It looks like the default max connections is 1024, is there any performance implication as we get closer to that limit?