celzero
Wireguard 0.55n has issues connecting if IP of Wireguard server has changed
I was a bit surprised to not find this here - the closest one might be https://github.com/celzero/rethink-app/issues/1538 . The older issue https://github.com/celzero/rethink-app/issues/1367 was something different.
Therefore let me add this here:
My Wireguard server is my local router (a Fritzbox) which connects to the Internet via a DynDNS service since the IP of my internet access usually changes every day.
ReThinkDNS has an issue with finding the Wireguard server then, which results in a stalled Wireguard connection (no data comes in) and makes "Always-on" unusable. Toggling either "total blockage" or the Wireguard connection itself usually fixes the issue - until the IP of the Wireguard server changes again. ~~Until now, I failed to find a task on the phone which, when excluded from the Wireguard connection, would solve the issue. (Judging from AfWall+, I would have expected I need to exclude ReThinkDNS and/or the system DNS service from Wireguard, but that does not change anything) As far as you can tell from the DNS protocol it seems ReThinkDNS does a query of the WG server but is happy if it finds it in its cache and does not care further. "DNS Amplifier" is off, "Never proxy DNS" either since that option does too much since it excludes all DNS entries from cache and Wireguard proxy. DynDNS URL of WG server is added as trusted domain.~~
Edit: corrected error description (see below for details): ReThink does get the new IP of the Wireguard server from the DNS, but it ignores the change and does not connect to Wireguard. That also explains why e
Expected behaviour would be that ReThinkDNS will
- do a DNS query of the defined Wireguard server if it detects a lack of answers from that server and if that server is a URL that you can query and ignore the DNS cache in that case
- and then try to reconnect to that server if the IP has changed..
Here is how to reproduce: a) phone is running and the WG server changes its ip address:
- set up / activate your wireguard tunnel, defining the endpoint as a URL (I use simple mode with total blockage)
- change the IP address (but not the DNS name) of the WG server
- Wireguard within ReThinkDNS will stall, seemingly no automatic attempt to reconnect will be made (I guess however ReThink keeps answering its own DNS query from its cache only for some reason). You need to manually disable Wireguard and restart it to get it running again.
b) WG server IP changes while phone is off:
- set up / activate your wireguard tunnel, defining the endpoint as a URL(I use simple mode with total blockage)
- set up your phone to request SIM card PIN after boot (should block mobile network access until phone is booted)
- shutdown phone
- change the IP address (but not the DNS name) of the WG server
- start phone, let it rest about a minute to let it finsh booting
- enter SIM card PIN and confirm. Phone will connect to mobile network now.
- Wireguard within ReThinkDNS will stall, seemingly no automatic attempt to reconnect will be made. You need to disable Wireguard and restart it to get it running again. If you are lucky, toggling the "total blockage" button off for 1...2 seconds and on again will do the job as well.
Thanks. I can see why this may happen.
Usually though, we expect dynamic DNS records changes to coincide with network changes (need not be the case, but usually is). If so, the endpoint domain is re-queried for IP.
We'll try to re-query after TTL expires.
@ignoramous if you expect DNS record changes to coincide with network changes, that is indeed a possible explanation. But unless I overlook something, this means it will basically never work if your Wireguard server depends on a DynDNS service:
- if your phone is connected via your local WiFi area and the router that connects the server gets a different IP, the network stays the same, but the IP changes
- if your phone is connected via another Wifi area, the same happens because the phone is still in the same Wifi network
- if your phone is connected to the mobile network, the same happens again because the phone is still in the same mobile network
- even if your phone is off and reconnects to the same network (Wifi or mobile) as before, RethinkDNS seems to treat this as "still the same network" and ignores the fact that the network was gone in between, hence the same applies there as well.
- if you put your phone into flight mode and then turn flight mode off again (which is usually the recommended solution to flush the DNS cache on a mobile phone), ReThinkDNS will prevent that since the phone reconnects to the same network.
Would it be possible to add some setting which turns this behaviour off and / or accept "no network" (flight mode) and any phone boot as a network change, even if if the network ID the phone reconnects to is the same?
if your phone is off and reconnects to the same network (Wifi or mobile) as before, RethinkDNS seems to treat this as "still the same network"
Not really. Whether or not 2 things are the "same network" actually depends on how Android reports it to Rethink. Rethink cannot and does not treat different networks as "same network" unless Android tells it to.
(which is usually the recommended solution to flush the DNS cache on a mobile phone)
Strange. Recommended by whom? There are a couple of ways to flush DNS caches (reddit / mirror).
possible to add some setting which turns this behaviour off and / or accept "no network" (flight mode) and any phone boot as a network change
It is possible to add new settings but since only power users use dyndns, I am reluctant to add it, especially since the major complaint by users of the app is the existing settings introducing complexity.
Today, Rethink will reconnect/re-establish WireGuard tunnel if you tap on the "Refresh" icon at the top right-hand corner of the Configure -> Proxy screen. This refresh should also trigger a re-evaluation of Peer (endpoint) names, if any.
Other than that, I think re-querying once the DNS answer time-to-live expires should be enough?
Strange. Recommended by whom? There are a couple of ways to flush DNS caches (reddit / mirror).
Chip.de and Adguard for example, just as the first two results a quick search for "clear dns cache android" gave out: https://praxistipps.chip.de/android-dns-cache-loeschen-so-gehts_172797 https://adguard-dns.io/kb/public-dns/solving-problems/how-to-flush-dns-cache/
But there are numerous others as well.
It is possible to add new settings but since only power users use dyndns, I am reluctant to add it, especially since the major complaint by users of the app is the existing settings introducing complexity.
Yes, that is a valid objection. Such a setting would need to be in some "extended settings" are which ReThink does not have today.
Today, Rethink will reconnect/re-establish WireGuard tunnel if you tap on the "Refresh" icon at the top right-hand corner of the Configure -> Proxy screen. This refresh should also trigger a re-evaluation of Peer (endpoint) names, if any.
Okay, I admit I have never noticed that one. I would have expected such an icon to be in the Proxy connection screen since "configuration" means for me "settings only"
while all connections are activated and deactivated in this screen:
I just tried that one, it does not help - see below.
Other than that, I think re-querying once the DNS answer time-to-live expires should be enough?
I fear I need to rephrase the issue, thanks for forcing me to take another, deeper look.
This is what I get:
DNS (done from my computer to get the TTL values): From the Adguard DNS server which I told Rethink to use I get from dig:
;; ANSWER SECTION:
aetherfunk.ddnss.de. 30 IN A 93.195.226.5
If I use the DNS server 8.8.4.4 Rethink obviously uses on its own after phone boot at times, I get
;; ANSWER SECTION:
aetherfunk.ddnss.de. 112 IN A 93.195.226.5
which means the entry should be re-queried after less than two minutes at max. ReThink seems to only pay limited care however - after I reconnected, it took about six minutes until the DNS query entry showed the current IP (unclear how old the cached entry was though, so the real TTL ReThink uses might be longer) - and interestingly enough, the DNS protocol shows all entries - both old and new - as "resolved by Cache". This means, I misjudged that and you cannot trust the "resolved by Cache" notice since the new IP must to have been resolved from somewhere else than the cache unless you did some real magic :-)
But that is not not the only problem I noticed (and this is what I never noticed before): even if ReThink has learned of the changed IP, the Wireguard connection still does not react. Instead, it stays stuck in "waiting" - "failing" - "waiting" - "failing". This state did not change for the last 45min. Using the button you suggested did not help at all.
In the meantime, the DNS entry gets polled every 2 minutes. This tells me it is actually not the DNS that fails but the Wireguard reconnection itself.
In this state, I can
- toggle the "total blockage" button within the Wireguard connection off/on which for some reason helps somtimes, but in other cases only resets the Upload/Download counter in the Wireguard Proxy view
- toggle the Wireguard connection itself off/on which then usually resolves the issue but lets all waiting connections briefly get out of the VPN tunnel.
Okay, after keeping an eye on this for some more days I updated the error description in the first post. The problem is that Rethink does not seem to update its Wireguard module after a DNS change of the Wireguard server. If you happen to have some new version that might solve this, I will be happy to test it :-)
I was the original poster for fixing DDNS and many issues about it. I thought it was fixed with #1013 but I was mistaken. Like the OP said, if the IP changes of the server the Rethink dosent notice this change because it keeps trying to connect to old one.
"Other than that, I think re-querying once the DNS answer time-to-live expires should be enough?"
Like you said, this should be what we are looking for. This would fix the problem. Reavulate the DNS of the endpoint on TTL end. Thanks so much for fixes from before, hope this can be fixed for the next version. This would make DDNS complete.
The problem is that Rethink does not seem to update its Wireguard module after a DNS change of the Wireguard server. Like you said, this should be what we are looking for. This would fix the problem. Reavulate the DNS of the endpoint on TTL end.
Thanks, both of y'all. Will attempt to fix this by re-querying on expired DNS answer TTLs.
This means, I misjudged that and you cannot trust the "resolved by Cache" notice since the new IP must to have been resolved from somewhere else than the cache unless you did some real magic
Could be a bug, but the Caching layer in Rethink can and does updates its (more popular) values in the background. That is, clients may get the latest value from the cache as it may have re-queried for it on its own.
@iulko thanks for confirming!
The problem is that Rethink does not seem to update its Wireguard module after a DNS change of the Wireguard server. Like you said, this should be what we are looking for. This would fix the problem. Reavulate the DNS of the endpoint on TTL end.
Thanks, both of y'all. Will attempt to fix this by re-querying on expired DNS answer TTLs.
I might misunderstand the last sentence, but if the Wireguard module is not "informed" about an IP change that the DNS module of Rethink has detected, letting the DNS module re-query would not help either, or am I getting that wrong? If the IP address of Wireguard server entry has changed, the Wireguard module of Rethink should get this information, and die Wireguard connection should be reset and reactivated using the new IP - this is what does not seem to happen today.
This means, I misjudged that and you cannot trust the "resolved by Cache" notice since the new IP must to have been resolved from somewhere else than the cache unless you did some real magic
Could be a bug, but the Caching layer in Rethink can and does updates its (more popular) values in the background. That is, clients may get the latest value from the cache as it may have re-queried for it on its own.
One side note: if the Wireguard module of Rethink is informed about DNS changes based on the TTL, this will solve issues if the phone has been offline for a longer time.
For cases where the Wireguard server is behind a dynamic DNS address though (which applies to every Wireguard server behind a standard ADSL connection) and the internet connection of the server is reset (e.g, by restarting the DSL modem), the Wireguard server will get a new IP regardless on how long the TTL is that has been communicated by the DNS before. Rethink will therefore believe the DNS to be valid even if it is not any more. Relying on DNS answers to be expired would not therefore work in this case.
For such cases, in addition of the above, a re-query of the Wireguard server IP should be forced whenever the Wireguard connection is detected dead (= red in connection view). If the IP address of Wireguard server entry has changed, the Wireguard connection should be reset and reactivated as well.
Might be worth adding this to https://github.com/celzero/rethink-app/issues/1864
I might misunderstand the last sentence, but if the Wireguard module is not "informed" about an IP change that the DNS module of Rethink has detected, letting the DNS module re-query would not help either, or am I getting that wrong? If the IP address of Wireguard server entry has changed, the Wireguard module of Rethink should get this information, and die Wireguard connection should be reset and reactivated using the new IP - this is what does not seem to happen today.
The system DNS is informed about the change but wireguard dosent re-query so there is no reason for system DNS to query that domain, if its gonna re-query than its gonna work because its gonna refresh the cache, that is easy.
For cases where the Wireguard server is behind a dynamic DNS address though (which applies to every Wireguard server behind a standard ADSL connection) and the internet connection of the server is reset (e.g, by restarting the DSL modem), the Wireguard server will get a new IP regardless on how long the TTL is that has been communicated by the DNS before. Rethink will therefore believe the DNS to be valid even if it is not any more. Relying on DNS answers to be expired would not therefore work in this case.
For such cases, in addition of the above, a re-query of the Wireguard server IP should be forced whenever the Wireguard connection is detected dead (= red in connection view). If the IP address of Wireguard server entry has changed, the Wireguard connection should be reset and reactivated as well.
If I undestand you correctly, and I think I do.
Well DNS works on TTL, you need to setup it to the lowest possible value and normally DNS system check for change only after TTL expired, thats the correct way it works in every website etc.
So even if you send a new IP to your DDNS provider your phone DNS will only update it after TTL expired because of how cache works.
Maybe it could be forced to get new value for the domain regardless of TTL but update of the IP on your server takes time, so we cant try to get new value right away (it would still get old value). So what time after we should try?
The best option is to try check if IP changed/re-query every x time. (This cant be only checked one time because as I said IP changed can take time, sometimes minutes if your ISP have some problem)
This time could be hard coded or we could just use TTL which is the value that is used in DNS systems.
So I propose for DDNS/wireguard to re-query for new IP when it can't connect exery X time, and X is TTL of the DNS record.
This way it will try to find new IP many times until its updated. Sorry to make it even a little more complicated @ignoramous :(
The system DNS is informed about the change but wireguard dosent re-query so there is no reason for system DNS to query that domain, if its gonna re-query than its gonna work because its gonna refresh the cache, that is easy.
[...]
So even if you send a new IP to your DDNS provider your phone DNS will only update it after TTL expired because of how cache works.
You are right, I should have checked my own findings once more before writing :-) As you can see above, the DNS TTL communicated from my provider is less than two minutes in any case - well within what would be acceptable for a reconnect.
So I propose for DDNS/wireguard to re-query for new IP when it can't connect exery X time, and X is TTL of the DNS record.
Yes, that should solve it. Agree! That should make the "Always-on" button usable for these cases as well, which would be great :-)
@Braintoe @iulko I don't have any WireGuard setups with DynDNS so I don't know if the changes we made for v055o and v055p fixed connectivity to Peers over DynDNS. If you folks have got the time and the appetite for risk, then please try v055p (which is a pre-release) and let me know how it goes: https://github.com/celzero/rethink-app/releases/tag/v0.5.5p (should also be available on F-Droid in a week or so, but it will not be released to the PlayStore).
@ignoramous I downloaded 0.55p and replaced my F-Droid RethinkDNS with it. Thanks for the work of you and your team on this :-)
First thing I noticed: the backup/restore obviously does not seem to take care about Wireguard tunnels, local blocklists and the "IP version" setting in the network section. Unsure if that is something that was caused by backing up by restoring though, therefore I will not ooen a separate issue for that now.
Second thing I noticed: if I set the IP version correctly, Dual stack Wireguard tunnels work a hell of a lot better than before, and initial connects (without a changed IP) are much faster.
All in all, the new version looks promising. Now let's see how Rethink reconnects when the IP of the server changes. I will post the result tomorrow!
Okay, a bit later than expected, but here is my test result: connection to Wireguard works flawlessly with 0.55p, hooray!
There is however a new bug - ReThink ignores the Wireguard DNS now whenever that one does not deliver a satisfactory result. but that is a new issue
I am on 0.55t I am connected to same wifi that looses the IP connection for some minutes because I disconnect the modem. Its in constant "Waiting" after that. Changing to LTE will make it work instantly, the Domain is propably updated than because of change of connection type.
I tried the other way to be on LTE to not be on the same connection that gets cut. Same problem. Constant "waiting" even after the domain gets updated IP.
@Braintoe Does wireguard reconnect for you after some minutes if you change IP of the Domain and dont do anything (dont reconnect your wifi or change to LTE, just let it sit)? (try 0.55t to be consistent)
@ignoramous Do I see right in the code that it tries every 2 minutes? Or is it from TTL of the domain
EDIT: But other than that I think Wireguard is much snappier and better, thank you. Like before even changing WIFI->LTE would hang the wireguard, but now its working 100% of the time. So before there were alot of problems, now I see only this one.
@ignoramous Do I see right in the code that it tries every 2 minutes? Or is it from TTL of the domain
You'll have to point me to the code you're seeing. If I recall, it is a mix: resolves endpoint address on network changes, on protocol changes, on TTL expiry etc (of the last one, I'm not sure).
tried the other way to be on LTE to not be on the same connection that gets cut. Same problem. Constant "waiting" even after the domain gets updated IP.
I couldn't understand. May be this scenario is worth fixing. Will you reword this please?
@Braintoe Does wireguard reconnect for you after some minutes if you change IP of the Domain and dont do anything (dont reconnect your wifi or change to LTE, just let it sit)? (try 0.55t to be consistent)
Just tested with 0.55p (sorry, no time to update at the moment) and advanced Wireguard. My "test protocol:
- reconnect router to get a new dynamic IP,
- wait about 2min
- open phone and try to access site or open an app that access the internet (to make sure Rethink is triggered and not simply sleeping)
- repeat step 2 and 3 five times
- open Rethink to see Wireguard state
Result: Wireguard state: "failing".
Then I disconnected Wireguard in Rethink and reconnected it - the connect failed, no answer.
Only after toggling Wifi, waiting a whille and then doing a manual reconnect the connection came back to life.
This case only occurs rarely here which is why I did not notice it yet.
I will try to repeat this test after upgrading. But I want to wait for 0.55u before doing that (one of the bugs mentioned here - stupid me did not note which one however...)