vpn(sshuttle) exit abnormally: exit status 99

[little-hang]

Describe the bug

1. 启动ktctl connect as this cmd：ktctl --debug -i registry.cn-hangzhou.aliyuncs.com/rdc-incubator/kt-connect-shadow:v0.3.6 -c /data/_qa/***/.kube/config --mode sshuttle --namespace qatest connect

Log 6:34PM DBG Background task log to /tmp/kt-76197548 6:34PM INF Using cluster context [email protected] (cluster.local) 6:34PM INF KtConnect 0.3.6 start at 30344 (linux amd64) 6:34PM DBG Rectify pod kt-rectifier-sxsbu created 6:34PM INF Fetching cluster time ... 6:34PM INF Fetching cluster time ... 6:34PM DBG Execute command [date +%s] in kt-rectifier-sxsbu:standalone 6:34PM DBG Time difference is -33 6:34PM INF Using sshuttle mode 6:34PM DBG Find 1 kt pods 6:34PM DBG Found local domains: .com 6:34PM DBG Private Key generated 6:34PM DBG Public key generated 6:34PM INF Successful create config map kt-connect-shadow-bwiab 6:34PM INF Deploying shadow pod kt-connect-shadow-bwiab in namespace qatest 6:34PM INF Waiting for pod kt-connect-shadow-bwiab ... 6:34PM INF Pod kt-connect-shadow-bwiab is ready 6:34PM DBG Service CIDR are: [10.233.0.0/16] 6:34PM DBG Pod CIDR are: [10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:34PM DBG Using cluster IP 10.31.77.15 6:34PM DBG Cluster CIDR are: [10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:34PM DBG Using port 63683 6:34PM DBG Request port forward pod:22 -> local:63683 via https://10.31.77.:6443 6:34PM INF Port forward local:63683 -> pod kt-connect-shadow-bwiab:22 established 6:34PM DBG Task vpn(sshuttle) with args [sshuttle --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.kt/key/kt-connect-shadow-bwiab.key --remote [email protected]:63683 --exclude 127.0.0.1 10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:34PM INF Setting up dns in local mode 6:34PM DBG Service found: aidd-auto-test.qatest 10.233.15.159 6:34PM DBG Dump hosts successful 6:34PM DBG Using port 1683 6:34PM DBG Request port forward pod:53 -> local:1683 via https://10.31.77.*:6443 6:34PM DBG Service aidd-auto-test added 6:34PM INF Port forward local:1683 -> pod kt-connect-shadow-bwiab:53 established 6:34PM INF Setup local DNS with upstream [tcp:127.0.0.1:1683 udp:10.10.10.10:53] 6:34PM INF Creating udp dns on port 10053 6:34PM DBG Background task vpn(sshuttle) closed, exit status 99 6:34PM DBG Task /usr/sbin/iptables with args [iptables --table nat --insert OUTPUT --proto udp --dest 127.0.0.1/32 --dport 53 --jump REDIRECT --to-ports 10053] 6:34PM INF --------------------------------------------------------------- 6:34PM INF All looks good, now you can access to resources in the kubernetes cluster 6:34PM INF --------------------------------------------------------------- 6:35PM DBG Restarting sshuttle ... 6:35PM DBG Task vpn(sshuttle) with args [sshuttle --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.kt/key/kt-connect-shadow-bwiab.key --remote [email protected]:63683 --exclude 127.0.0.1 10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:35PM DBG Background task vpn(sshuttle) closed, exit status 99 6:35PM DBG Restarting sshuttle ... 6:35PM DBG Task vpn(sshuttle) with args [sshuttle --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.kt/key/kt-connect-shadow-bwiab.key --remote [email protected]:63683 --exclude 127.0.0.1 10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:35PM DBG Background task vpn(sshuttle) closed, exit status 99 6:35PM DBG Restarting sshuttle ... 6:35PM DBG Task vpn(sshuttle) with args [sshuttle --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.kt/key/kt-connect-shadow-bwiab.key --remote [email protected]:63683 --exclude 127.0.0.1 10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32] 6:35PM DBG Pod kt-rectifier-sxsbu deleted 6:35PM DBG Heartbeat port forward 1683 ticked at 2022-07-19 18:35:53 6:35PM DBG Heartbeat port forward 63683 ticked at 2022-07-19 18:35:55 6:35PM WRN Failed to lookup 3.centos.pool.ntp.org. (28) in dns (127.0.0.1:1683) error=EOF 6:35PM WRN Failed to lookup 3.centos.pool.ntp.org. (1) in dns (127.0.0.1:1683) error=EOF 6:35PM DBG Empty answer for domain lookup 3.centos.pool.ntp.org. (28) 6:35PM DBG Found domain 3.centos.pool.ntp.org. (1) in dns (10.10.10.10:53) 6:35PM DBG Background task vpn(sshuttle) closed, exit status 99 6:36PM DBG Restarting sshuttle ... 6:36PM DBG Task vpn(sshuttle) with args [sshuttle --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.kt/key/kt-connect-shadow-bwiab.key --remote [email protected]:63683 --exclude 127.0.0.1 10.233.0.0/16 10.233.0.0/16 10.31.0.0/16 10.232.206.35/32]

Environment (please complete the following information):

• OS: centos
• Kubernetes: v1.16.15
• KT Version: V0.3.6

[linfan]

这个问题不太好排查，exit status 99 在 sshuttle 里代表“未知错误“，就是运行环境检查都通过，但是进程异常奔溃。通常还是由于特定的环境问题，但没有现场的话不太好处理。

是什么原因不能使用默认的 tun 模式呢？