netclient icon indicating copy to clipboard operation
netclient copied to clipboard

[BUG] Client v0.20.1 cannot update endpointip

Open nitsujri opened this issue 2 years ago • 11 comments

Problem

The Endpoint IP for all clients were not automatically getting set and even trying to manually set them via UI wasn't working.

Cursor_and_root_ip-172-31-0-116___home_ubuntu

I recreated my entire AWS VPN bastion network using 0.20.1. My guess is based on the image above, there's an issue communicating with turn? I'm not familiar enough.

Workaround

Downgrade the client to 0.20.0 via install script and endpointip works great again.

Both netmaker and netmaker-ui can remain 0.20.1. So far haven't noticed any issues.

Expected Result

v0.20.1 works the same as v0.20.0 - endpointip is automatically updated and/or be manually updated via UI.

https://github.com/gravitl/netmaker/issues/2378 - Original issue that I opened thinking the problem was over there.

nitsujri avatar Jun 08 '23 09:06 nitsujri

what is output of docker inspect netmaker | grep STUN when run on netmaker server

mattkasun avatar Jun 08 '23 20:06 mattkasun

@mattkasun the output of that:

root@ip-172-31-50-8:/home/ubuntu# docker inspect netmaker | grep STUN
                "STUN_PORT=3478",
                "STUN_LIST=stun.netmaker.internal-wireguard.jenfi.com:3478,stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302",

nitsujri avatar Jun 08 '23 23:06 nitsujri

Also related: https://github.com/gravitl/netmaker/issues/2381

nitsujri avatar Jun 09 '23 03:06 nitsujri

I had the same issue with oracle cloud Arm Instance <-> AMD FreeTier Instance communication, but issue is on only the ARM machine. The wireguard host was visible (wg show) on the AMD machine, but not on the ARM machine. After downgrading the client to v0.20.0 as @nitsujri's solution and specifying the local IP, ping was worked.

miyu4u avatar Jun 10 '23 12:06 miyu4u

Same issue here. Netclient v0.20.5 is not recognizing change of external IP and seems not to update the host on netmaker server. I rolled back to v0.20.4 and will observe if this issue persists. Further, I can confirm netclient v0.20.0 is syncing the IP properly.

pernetz avatar Aug 04 '23 06:08 pernetz

Same issue here. Netclient v0.20.5 is not recognizing change of external IP and seems not to update the host on netmaker server. I rolled back to v0.20.4 and will observe if this issue persists. Further, I can confirm netclient v0.20.0 is syncing the IP properly.

Have you had this issue on v0.20.4 yet?

senses3 avatar Aug 26 '23 17:08 senses3

Yupp same issue on v0.20.4 and also on latest release v0.20.6 I started digging deeper and detected tons of errors "wireguard_linux.go 146" in the log.

image

Also I could figure out the IP update is triggered by:

  1. rebooting machine
  2. service restart systemctl restart netclient
  3. using netclient pull command

However it is not picking up the change of IP on its own.

Any advice?

pernetz avatar Aug 26 '23 19:08 pernetz

Issue of not updating external IP persists further with v0.21.0 on several machines. I am wondering if this can be caused by DNAT setup? How does netclient catches the external IP?

@mattkasun Can you give any advice?

pernetz avatar Sep 12 '23 09:09 pernetz

Netclient uses stun to set the public ip

mattkasun avatar Sep 12 '23 10:09 mattkasun

With further testing I could figure out there is something strange since v.0.20.1. Netclient v0.20.0 is the last working version (for me), recognizing and pushing external IP to Netmaker server correctly.

  1. Tested on
  • Debian 11
  • Debian 12
  • Ubuntu 22.04 --> Result: OS has no influence on issue
  1. Tested with different DNS resolvers
  • Adguard Home active
  • Adguard Home inactive
  • using 8.8.8.8 and 1.1.1.1 directly --> Result: DNS has no influence on issue
  1. Outcome of journalctl -xe | grep netclient
  • Version v.0.20.0 image

  • Version v0.21.0 image

I am missing the "endpoint has changed" message

pernetz avatar Sep 15 '23 06:09 pernetz

Want to provide an update on this topic.

Bad news is that issue still exists on version 0.21.1 The good news, it only appears on Debian distros. Ubuntu is syncing as intended now.

Update 2: Redeployed a brand new installation of netmaker server and updated all clients to v0.21.1 --> endpoint update works properly. Not sure what caused that issue when update from a previous version.

Kudos to netmaker team! Great work, I love this awesome project.

pernetz avatar Oct 26 '23 18:10 pernetz