tikv-importer fails with no error message when using domain name as listening address
Bug Report
I was running tidb-lightning in Kubernetes with domain name as the listening address, but the tikv-importer failed with no logs. After some debugging, I found that tikv-importer can start using Pod IP as listening address.
To reproduce this, simply add tikv-importer binary to pingcap/alpine-glibc Docker image, and run the following commands in the new image. To help ease the reproduce, I've built an image uhub.ucloud.cn/pingcap/tidb-lightning:latest.
$ docker run -it --rm uhub.ucloud.cn/pingcap/tidb-lightning:latest
/ # cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 318b01eaf0c1
/ # /tikv-importer -h
TiKV Importer 3.0.2
The TiKV Authors
The importer server for TiKV
USAGE:
tikv-importer [OPTIONS]
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-A, --addr <IP:PORT> Set the listening address
-C, --config <FILE> Set the configuration
--import-dir <PATH> Set the directory to store importing kv data
--log-file <FILE> Set the log file
--log-level <LEVEL> Set the log level [possible values: trace, debug, info, warn, error, off]
/ # /tikv-importer --addr=318b01eaf0c1:20170
/ # echo $?
1
/ # /tikv-importer --addr=172.17.0.2:20170
^C[2019/08/11 05:48:33.494 +00:00] [INFO] [signal_handler.rs:21] ["receive signal 2, stopping server..."]
/#
To run tidb-lightning in Kubernetes, tidb-lightning needs to support domain name which is required for stateful applications.
This isn't a problem specific to Importer. TiKV has the same problem since SocketAddr::from_str won't (and shouldn't) do DNS resolution.
https://github.com/tikv/tikv/blob/f5215d631d3145d824c0d0940cd490ff3f8bbf80/src/server/server.rs#L104
The TiDB Operator works fine because the TiKV server is listening on 0.0.0.0. The DNS name is exposed through --advertise-addr.
/tikv-server --pd=c-pd:2379 --advertise-addr=pod_name-0.pod_name-peer.k8s_ns.svc:20160 --addr=0.0.0.0:20160 --data-dir=/var/lib/tikv --capacity=3667GB --config=/etc/tikv/tikv.toml
The --advertise-addr setting is meaningless for Importer since we did not register Importer on PD.
Lightning supports DNS names just fine.
bin/tidb-lightning --importer 318b01eaf0c1:20170 ...
The only issue here is "no error message" (TiKV will emit a FATAL log like below). I'm gonna transfer this issue to the Importer repo.
[2019/08/11 14:24:45.576 +08:00] [FATAL] [server.rs:263] ["failed to create server: AddrParseError(())"]