clickhouse-js icon indicating copy to clipboard operation
clickhouse-js copied to clipboard

Add support for multiple addresses same as in golang client

Open kobi-co opened this issue 1 year ago • 7 comments

Hi, It would be great if you'll be able to support multiple addresses same as in the golang client library for better support on working with clickhouse clusters.

Thank you!

kobi-co avatar Jul 05 '23 08:07 kobi-co

@kobi-co, it should be similar to https://github.com/ClickHouse/clickhouse-go#http-support-experimental, right?

http://host1:8123,host2:8123/database?dial_timeout=200ms&max_execution_time=60

slvrtrn avatar Jul 11 '23 20:07 slvrtrn

I actually meant the option to pass multiple addresses in the Open method, but if we can get the same result by passing one address with multiple hosts that's good as well.

Thanks!

kobi-co avatar Jul 11 '23 20:07 kobi-co

Plus 1

Hello

I have clickhouse cluster - 4 servers (4 shards, no replicas). Now, this client can connect only to one instance (For example - shard 2). IF shard 2 DOWN - nothing work ...

  1. pass multiple: { host, port, username, password, dbName }[]. AS ARRAY
  2. switching connection and other INSIDE lib

zibellon avatar Aug 29 '23 05:08 zibellon

even tough we can manage this by ourselves in application side, having this feature in client would be a lot better.

generic connection-string could be used.

I can work on this if you guys are open to PR

ayZagen avatar Sep 11 '24 22:09 ayZagen

The tricky part is probably the internal implementation with the Node.js client, as it looks like it will require multiple HTTP agents (due to KeepAlive and different hosts). Then, what is the strategy for choosing one of the available underlying agents—random/round-robin? Also, how will it work with a custom HTTP agent instance that can be provided in the client config? What if a particular node is down? And many other corner cases...

Another option is to keep the entire client implementation as-is and provide an auxiliary client builder for multiple addresses with multiple internal client instances inside. But this is also fairly convoluted.

Anyway, for a smooth client usage experience, it will be required to track the nodes that are down and so on, and that looks like an LB responsibility, which, in my opinion, should be used with clusters that expose the HTTP interface anyway (like nginx with a very simple config that we use in the client tests with multiple nodes).

CC @mshustov WDYT?

slvrtrn avatar Sep 12 '24 16:09 slvrtrn

Anyway, for a smooth client usage experience, it will be required to track the nodes that are down and so on, and that looks like an LB responsibility, which, in my opinion, should be used with clusters that expose the HTTP interface anyway (like nginx with a very simple config that we use in the client tests with multiple nodes).

Agreed, IMO it's the correct design: use a reverse proxy for load balancing, health monitoring, ssl termination, etc.

mshustov avatar Sep 12 '24 18:09 mshustov

By no means I am a Clickhouse expert but isn't clickhouse cluster topology active-active? If so, it wouldn't matter which instance is receiving the connection. For load balancing or query splitting end users can still use nginx like proxies, this feature won't prevent them from doing so.

As for the context of this request I believe ensuring one stable connection is enough. It could select randomly or one by one, really doesn't matter. There is no need to track each node just iterate until one connection succeeds.

ayZagen avatar Sep 12 '24 18:09 ayZagen