[v2] also-proxy is not working properly

[hrdcdr1337]

Hi, I've experienced issues with also-proxy feature on linux. After start the intercept also-proxy addresses are available but not respond on open ports.

How to reproduce

1. Check service availability without intercepts:

~ curl https://ifconfig.me
*.*.*.* (return my ip)

~ traceroute ifconfig.me
  1   192.168.1.1  1.392ms  1.151ms  0.852ms 
  2   *.*.*.*  1.662ms  1.247ms  2.614ms 
  3   10.245.138.241  1.478ms  1.399ms  1.230ms 
  4   10.245.138.242  2.129ms  2.189ms  1.498ms 
  5   178.49.128.50  2.389ms  2.662ms  2.373ms 
  6   72.14.215.165  47.926ms  46.576ms  44.528ms 
  7   72.14.215.166  47.421ms  47.314ms  47.300ms 
  8   108.170.250.33  43.908ms  43.355ms  43.339ms 
  9   108.170.250.51  47.317ms  46.471ms  46.942ms 
 10   216.239.51.32  60.353ms  60.487ms  60.267ms 
 11   172.253.66.108  60.729ms  60.543ms  59.959ms 
 12   142.250.56.221  57.622ms  57.691ms  57.296ms 
 13   *  *  * 
 14   *  *  * 
 15   *  *  * 
 16   *  *  * 
 17   *  *  * 
 18   *  *  * 
 19   *  *  * 
 20   *  *  * 
 21   *  *  * 
 22   34.117.59.81  57.442ms  57.497ms  57.177ms

2. Add also-proxy for https://ifconfig.me/ to ~/.kube/config

~ dig ifconfig.me
...
;; ANSWER SECTION:
ifconfig.me.		573	IN	A	34.117.59.81
...

apiVersion: v1
clusters:
- name: dev-utp-apps
  cluster:
    certificate-authority-data: ****
    server: ****
    extensions:
    - name: telepresence.io
      extension:
        also-proxy:
        - 34.117.59.81/32

3. Run intercept or connect

~ telepresence connect
Launching Telepresence Root Daemon
Launching Telepresence User Daemon
Connected to context dev-utp-apps (https://*.*.*.*:6443)

~ telepresence status 
Root Daemon: Running
  Version   : v2.4.3 (api 3)
  DNS       :
    Remote IP       : 10.43.0.10
    Exclude suffixes: [.arpa .com .io .net .org .ru]
    Include suffixes: []
    Timeout         : 4s
  Also Proxy: (1 subnets)
    - 34.117.59.81/32
User Daemon: Running
  Version           : v2.4.3 (api 3)
  Ambassador Cloud  : Logged out
  Status            : Connected
  Kubernetes server : https://*.*.*.*:6443
  Kubernetes context: telepresence-testing-cluster
  Telepresence proxy: ON (networking to the cluster is enabled)
  Intercepts        : 0 total

4. Check the same as before:

~ curl https://ifconfig.me
curl: (7) Failed to connect to ifconfig.me port 443: Connection refused

~ traceroute ifconfig.me
traceroute to ifconfig.me (34.117.59.81), 64 hops max
 1   34.117.59.81  0.003ms  0.002ms  0.002ms 

~ ping ifconfig.me
PING ifconfig.me (34.117.59.81) 56(84) bytes of data.
64 bytes from 81.59.117.34.bc.googleusercontent.com (34.117.59.81): icmp_seq=1 ttl=64 time=0.088 ms
64 bytes from 81.59.117.34.bc.googleusercontent.com (34.117.59.81): icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 81.59.117.34.bc.googleusercontent.com (34.117.59.81): icmp_seq=3 ttl=64 time=0.085 ms
^C
--- ifconfig.me ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.078/0.083/0.088/0.004 ms

There is no problem when connecting to subnets directly w/o also-proxy feature

[thallgren]

The ping command will not work because it uses ICMP which is not supported by telepresence VIF. It currently supports TCP and UDP only. It's odd that curl doesn't work though. What's the output of:

kubectl run curl-from-cluster --rm -it --image=docker.io/pstauffer/curl --restart=Never -- curl --silent http://ifconfig.me

[hrdcdr1337]

hey @thallgren thank you for rapid answer. The command you provided works well and return server ip

kubectl run curl-from-cluster --rm -it --image=docker.io/pstauffer/curl --restart=Never -- curl --silent http://ifconfig.me
x.x.x.x (server ip)
pod "curl-from-cluster" deleted

[modax]

Looks like the issue is when /32 subnet is specified for also-proxy.

E.g., I want to access IP 10.118.0.3. If I add '10.118.0.3/32, subnet is present in ip addr:

ip addr show tel0

6400: tel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none 
    inet 10.11.0.0/20 scope global tel0
       valid_lft forever preferred_lft forever
    inet 10.118.0.3/32 scope global tel0
       valid_lft forever preferred_lft forever
    inet 10.11.5.0/24 scope global tel0
       valid_lft forever preferred_lft forever
    inet 10.11.4.0/24 scope global tel0
       valid_lft forever preferred_lft forever
    inet6 fe80::3506:3f7f:dd37:c3d0/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

But it is not present in ip route: ip route | grep tel

10.11.0.0/20 dev tel0 proto kernel scope link src 10.11.0.0 
10.11.4.0/24 dev tel0 proto kernel scope link src 10.11.4.0 
10.11.5.0/24 dev tel0 proto kernel scope link src 10.11.5.0

Adding ip route manually does not help. Generally, for anyone reading this, your best bet is to add such a subnet to also-proxy where the first address and the last address does not match the IP you want to access. Thus 10.118.0.0/24 works. For some odd reason 10.118.0.2/31 also makes 10.118.0.3 work (despite it being the last address in the subnet) while 10.118.0.0/30 does not. So it is a bit hit and miss.

[cindymullins-dw]

We're not seeing these issues or reports of them recently. If it continues to be an issue please let us know.