dnsmonster icon indicating copy to clipboard operation
dnsmonster copied to clipboard

ClickHouse over network

Open HesaMtiPu opened this issue 2 years ago • 8 comments

I made a small change that leverages builtin driver's support. Most likely I'll make a change to the way the Address is provided so we can have different credentials and TLS support for each address later on. Please let me know if the latest commit solves your issue. Happy to re-open if it doesn't

Originally posted by @mosajjal in https://github.com/mosajjal/dnsmonster/issues/27#issuecomment-1144723538

HesaMtiPu avatar Jun 13 '22 07:06 HesaMtiPu

Hi !!! Is there any way that we mention only one click-house IP in dnsmonster and click-house or zookeeper can manage the replication itself and replicate the data to other node. Because if we mention multiple values and IP, then what is the need of click-house replication, we can do that in standalone mode. So if you have any other way, just mention one IP of click-house in dnsmonster and let click-house manage the data replication itself. Thanks !!!

HesaMtiPu avatar Jun 13 '22 07:06 HesaMtiPu

Hey mate. I think best way to solve your issue is to set up a Kafka in the middle. Unfortunately Clickhouse-go doesn't support HTTP endpoint so I can't provide that option in dnsmonster just yet, but when that happens you'll have another option to solve data replication. There's no easy way to achieve your requirement without messing up the whole design.

mosajjal avatar Jun 13 '22 08:06 mosajjal

Hi !! Actually I was saying this because we configure dnsmonster and clickhouse on different VMs means a different VM for both. I want to keep dnsmonster separate from clickhouse so that the load is less and there is less chance of server down. That's why I want this kind of configuration. by the way thanks for your reply and please let me know if you have any other options in dnsmonster. Thanks!!!

HesaMtiPu avatar Jun 14 '22 03:06 HesaMtiPu

Oh right I think I get your point. If you want to put dnsmonster and ClickHouse in different VMs, that's totally fine. use the option --clickhouseAddress= and point it to the IP address of your ClickHouse VM and it should all work fine. Hopefully this answers your question.

mosajjal avatar Jun 14 '22 09:06 mosajjal

yes I know this and I am using it also but our question comes again at the same place that if node with given clickhouse Ip goes down then my replication will stop and if we give multiple values ​​then clickhouse replication What is the requirement of . Simply I want that if the IP of the clickhouse node given in dnsmonster is down, then dnsmonster itself will continue the replication through another node in clickhouse replication mode.

HesaMtiPu avatar Jun 14 '22 10:06 HesaMtiPu

If you're looking into setting the strategy of multiple ClickHouse endpoints (load balance, fail-over etc), I'm afraid that's not easily doable with the current native protocol. Hopefully with the addition of ClickHouse on HTTP (which is on the horizon) we can enable this feature. Take a look at this article for more info.

mosajjal avatar Jun 14 '22 10:06 mosajjal

YA !!! I know it's not that easy. But we are trying to achieve this requirement in some way.

HesaMtiPu avatar Jun 15 '22 04:06 HesaMtiPu

#71

mosajjal avatar Jun 24 '23 00:06 mosajjal