tikv
How to use SDK to access RawKV from outside the k8s cluster?
We have deployed a TiKV cluster on k8s, which can only be accessed from within the k8s cluster. In order to enable access to the TiKV cluster from outside the k8s cluster, we need to configure CoreDNS resolution on each client machine, but this is not elegant. Is there a way to specify the DNS for accessing the k8s internal cluster through the application layer?
Can we implement DNS resolution within the SDK without modifying the system's DNS settings? Would this be a good approach?
This issue is moved by https://github.com/pingcap/tidb-operator/issues/5051
I am not familiar with k8s, but I think maybe you can bind a domain name for each tikv node and PD node so that they can be accessed from outside the k8s cluster. However, exposing internal addresses may raise security concerns. A safer approach may be to start a rawkv proxy service within the k8s cluster to forward rawkv requests.
Currently, we are using proxy forwarding, but it is not suitable for scenarios with high-performance requirements and limited resources. In the current Tikv on k8s solution, each Tikv has its own domain name, but this domain name can only be resolved through a dedicated DNS server. So I am thinking about adding a configuration DNS server function for the client so that there is no need to change the system configuration to resolve the domain name.
This idea sounds interesting to me. I did some research and found that grpc provides the ability to customize name resolver functions (https://github.com/grpc/grpc-go/blob/master/examples/features/name_resolving/README.md) If possible, I think we can add an option to the client to support a custom name resolver.
Writing a comment as also facing this issue. We would like to use TiKV on EKS for JuiceFS, which uses just the RawKV API, from outside the cluster, but this is not stiraghtforward, at least with how TiDB operator likes to set things up.
