algo
algo copied to clipboard
trailofbits
Not able to view Twitter Profiles and Tweet Conversations/Threats on iOS
OS / Environment (where do you run Algo on)
t2.micro Linux/UNIX Ubuntu
Cloud Provider (where do you deploy Algo to)
AWS
Summary of the problem
When connected to my AlgoVPN and using the Twitter app on iOS, loading user profiles, viewing conversations or threats as well as members of a conversation is not possible. Any other page or app seems to work without issues. I doubt it has to do with connection strength even though I am based in China... But any help would be appreciated.
Steps to reproduce the behavior
Loading user profile
- While connected to AlgoVPN, open Twitter on iOS.
- Click on any name of the user to see their profile
- I get a blank profile page consisting of nothing but the user's Twitter handle and a message saying "Something went wrong, try again.", see picture below
Loading conversation/threat
- While connected to AlgoVPN, open Twitter on iOS.
- Find a tweet with a conversation/replies or with a threat.
- Click on tweet to open conversation/threat
- I get a page only opneing the original tweet without anything more
On other VPNs (commercial or corporate) Twitter App on iOS works perfectly fine and without above issues.
I've run into this problem in the past when using Twitter in a browser and I think it's due to Twitter blocking connections from certain cloud provider networks. But for some reason I no longer have this problem even though I'm also using Amazon (in my case Lightsail in region us-east-1b).
I've never encountered this problem when using Tweetbot on either iOS or macOS.
@domdewom What service provider did you use?
I use DigitalOcean and have had the same instance running for about 2 years now, and in the last week or so I've had a tonne of issues.
I don't think it's related to certain blocking of cloud provider networks as it'll work randomly, but rather some something else.
The reason I say that is that the failure of twitter images (and just CDN served assets in general) is that they will eventually work if for example you restart the browser, or even refreshing the page a few times.
The network traffic is failing to do DNS lookups, rather than connecting and being blocked.
ERR_NAME_NOT_RESOLVED is the error I always see in the traffic for domains like pbs.twimg.com
Nothing has changed as far as I know so I think maybe just recreating a new instance.
twitter is definitely blocking certain cloud provividers.
@davidemyers - I just switched from ec2 in ue1 to lightsail ue2, still get the same result. are you doing anything else differently? I'm also blocking ads on dns. maybe that's it?
@mikedizon I've never used Algo's ad blocking feature so that hasn't changed. I use Quad9 for DNS instead of Cloudflare, but that also hasn't changed as I've been using Quad9 for a long time.
@davidemyers i don't think i saw that as an option when running ./algo when does that appear?
To change the secure DNS servers you use before you deploy, edit config.cfg and change dnscrypt_servers. On a deployed server edit /etc/dnscrypt-proxy/dnscrypt-proxy.toml and change server_names.
At the moment I'm actually using both Quad9 and Cloudflare with malware blocking, so my dnscrypt-proxy.toml contains:
server_names = ['quad9-doh-ip4-port443-filter-pri', 'quad9-doh-ip6-port443-filter-pri', 'cloudflare-security', 'cloudflare-security-ipv6']
@davidemyers updated my config.cfg file with:
ipv4:
- cloudflare-security
- quad9-doh-ip4-port443-filter-pri
ipv6:
- cloudflare-security-ipv6
- quad9-doh-ip6-port443-filter-pri
Not sure if it's a coincidence, but Twitter is no longer blocking traffic from my devices!
FWIW, I also am seeing a lot of recent issues in past week or so and similar behavior to what @samkelleher described. I am on Digital Ocean in NYC3 with adblock and haven't changed anything in several months. I also see that a page or app will not load, but then after some refreshing or closing/reopening, it will actually work (but may take a few tries). So it certainly smells of some kind of DNS problem, but I am not sure what. The only other odd clue I noticed is the issue is almost entirely on mobile/tablet devices. I do not think I have encountered the problem on desktop. Which I do not understand.
I was going to try this weekend to setup an instance on DO/NYC3 without adblock and see if that makes any difference.
@iamvishnurajan It's starting to sound like the problem is with the Cloudflare DNS servers used by Algo by default, but that maybe the alternate Cloudflare servers with malware domain blocking are OK.
I suggest those having issues try using cloudflare-security and cloudflare-security-ipv6 instead of cloudflare and cloudflare-ipv6.
I should add that on my phone, Twitter works fine on the web, but the native app gets blocked.
Using google, images, and styles on some sites (including github) were not loading.
I updated the /etc/dnscrypt-proxy/dnscrypt-proxy.toml on my instance to cloudflare-security and cloudflare-security-ipv6, and rebooted it. I will run this for a bit and report back.
I noticed this behavior too, not just with Twitter, but also with the iOS App Store, Discord, NPR, Google Maps, google web searches, and various websites not loading images or other assets.
I enabled logging for dnscrypt-proxy on my Algo server, and found many RESPONSE_ERROR response codes for seemingly random host names. Here's a selection from this morning:
[2021-09-17 13:50:04] 10.49.0.2 init.itunes.apple.com PASS 15ms cloudflare-security
[2021-09-17 13:50:04] 10.49.0.2 init.itunes.apple.com A PASS 17ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 configuration.ls.apple.com A PASS 0ms -
[2021-09-17 13:50:05] 10.49.0.2 configuration.ls.apple.com PASS 16ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 api.smoot.apple.com PASS 14ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 api.smoot.apple.com A PASS 15ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 e10499.dsce9.akamaiedge.net RESPONSE_ERROR 18ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 bag-smoot.v.aaplimg.com A RESPONSE_ERROR 20ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 bag-smoot.v.aaplimg.com RESPONSE_ERROR 21ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 click.fourhourmail.com RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:05] 10.49.0.2 e1329.g.akamaiedge.net PASS 0ms -
[2021-09-17 13:50:06] 10.49.0.2 www.allthehacks.com A PASS 17ms cloudflare-security
[2021-09-17 13:50:06] 10.49.0.2 www.allthehacks.com PASS 103ms cloudflare-security
[2021-09-17 13:50:06] 10.49.0.2 domains.podpage.com PASS 41ms cloudflare-security
[2021-09-17 13:50:06] 10.49.0.2 xp.apple.com A RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:06] 10.49.0.2 xp.apple.com RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:06] 10.49.0.2 r3.o.lencr.org PASS 0ms -
[2021-09-17 13:50:06] 10.49.0.2 r3.o.lencr.org A PASS 26ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 a1887.dscq.akamai.net RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 guzzoni.apple.com RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 guzzoni.apple.com A RESPONSE_ERROR 16ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 ocsp2.apple.com PASS 17ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 ocsp2.apple.com A PASS 18ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 stackpath.bootstrapcdn.com A RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 stackpath.bootstrapcdn.com PASS 16ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 redwood-labs.s3.amazonaws.com A RESPONSE_ERROR 16ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 cdnjs.cloudflare.com A RESPONSE_ERROR 16ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 fonts.gstatic.com A PASS 0ms -
[2021-09-17 13:50:07] 10.49.0.2 cdnjs.cloudflare.com RESPONSE_ERROR 20ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 www.googletagmanager.com REJECT 0ms cloudflare-security
[2021-09-17 13:50:07] 10.49.0.2 www.googletagmanager.com A REJECT 0ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 fonts.gstatic.com RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 cdn.jsdelivr.net A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 fonts.googleapis.com A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 fonts.googleapis.com RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 s3.us-west-1.amazonaws.com RESPONSE_ERROR 18ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 s3.us-west-1.amazonaws.com A RESPONSE_ERROR 18ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 cdn.jsdelivr.net RESPONSE_ERROR 17ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 ajax.googleapis.com A RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 ajax.googleapis.com RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 ocsp2.g.aaplimg.com A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 ocsp2.g.aaplimg.com RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 a1845.dscg2.akamai.net RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 redwood-labs.s3.amazonaws.com PASS 105ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 gstaticadssl.l.google.com RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 s3-us-west-1-w.amazonaws.com RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 s3-us-west-1-w.amazonaws.com A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 player.simplecast.com A RESPONSE_ERROR 14ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 player.simplecast.com RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 allthehacks.substack.com RESPONSE_ERROR 12ms cloudflare-security
[2021-09-17 13:50:08] 10.49.0.2 allthehacks.substack.com A RESPONSE_ERROR 15ms cloudflare-security
[2021-09-17 13:50:09] 10.49.0.2 podpage.imgix.net PASS 20ms cloudflare-security
[2021-09-17 13:50:09] 10.49.0.2 podpage.imgix.net A PASS 43ms cloudflare-security
[2021-09-17 13:50:09] 10.49.0.2 dualstack.com.imgix.map.fastly.net A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:09] 10.49.0.2 dualstack.com.imgix.map.fastly.net PASS 17ms cloudflare-security
[2021-09-17 13:50:10] 10.49.0.2 client-api.itunes.apple.com A RESPONSE_ERROR 13ms cloudflare-security
[2021-09-17 13:50:10] 10.49.0.2 client-api.itunes.apple.com PASS 15ms cloudflare-security
The same request for a host name would sometimes succeed with PASS subsequent to a RESPONSE_ERROR failure (see the last two lines above).
I changed dnscrypt-proxy's config to use google instead of cloudflare or cloudflare-security, and the RESPONSE_ERROR response codes immediately stopped and everything works again. If I enable a cloudflare name server, the intermittent errors repeat.
I built a fresh Algo server this morning and was able to repeat the issue: using cloudflare name servers consistently results in RESPONSE_ERROR response codes.
Also reported this to the slack channel.
@quinncomendant thanks much for the tip - my issues were also not resolved after changing to cloudflare-security. I have updated mine to google now after your comment and at least on a first test, apps that were impossible to open in recent time can now be opened.
For the benefit of folks, and as @davidemyers noted above on an existing deployment, edit /etc/dnscrypt-proxy/dnscrypt-proxy.toml, and you can change the server_names to server_names = ['google', 'google-ipv6']. After this, restart the dnscrypt-proxy service with sudo systemctl restart dnscrypt-proxy and verify successful restart by checking the syslog (sudo tail /var/log/syslog).
If folks want to play with other server names (perhaps Quad9?) the list is here: https://dnscrypt.info/public-servers/. At least for now though, it certainly appears that cloudflare is problematic for some reason.
For the sake of completeness, dear reader, after editing dnscrypt-proxy.toml you need to restart it with sudo systemctl restart dnscrypt-proxy and then should peek at the syslog to see if it restarted successfully (sudo tail /var/log/syslog).
@quinncomendant thank you for completeness - edited my comment so folks are not left inadvertently hanging after only editing the .toml
I noticed this behavior too, not just with Twitter, but also with the iOS App Store, Discord, NPR, Google Maps, google web searches, and various websites not loading images or other assets.
I enabled logging for dnscrypt-proxy on my Algo server, and found many
RESPONSE_ERRORresponse codes for seemingly random host names. Here's a selection from this morning:[2021-09-17 13:50:04] 10.49.0.2 init.itunes.apple.com PASS 15ms cloudflare-security [2021-09-17 13:50:04] 10.49.0.2 init.itunes.apple.com A PASS 17ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 configuration.ls.apple.com A PASS 0ms - [2021-09-17 13:50:05] 10.49.0.2 configuration.ls.apple.com PASS 16ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 api.smoot.apple.com PASS 14ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 api.smoot.apple.com A PASS 15ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 e10499.dsce9.akamaiedge.net RESPONSE_ERROR 18ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 bag-smoot.v.aaplimg.com A RESPONSE_ERROR 20ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 bag-smoot.v.aaplimg.com RESPONSE_ERROR 21ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 click.fourhourmail.com RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:05] 10.49.0.2 e1329.g.akamaiedge.net PASS 0ms - [2021-09-17 13:50:06] 10.49.0.2 www.allthehacks.com A PASS 17ms cloudflare-security [2021-09-17 13:50:06] 10.49.0.2 www.allthehacks.com PASS 103ms cloudflare-security [2021-09-17 13:50:06] 10.49.0.2 domains.podpage.com PASS 41ms cloudflare-security [2021-09-17 13:50:06] 10.49.0.2 xp.apple.com A RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:06] 10.49.0.2 xp.apple.com RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:06] 10.49.0.2 r3.o.lencr.org PASS 0ms - [2021-09-17 13:50:06] 10.49.0.2 r3.o.lencr.org A PASS 26ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 a1887.dscq.akamai.net RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 guzzoni.apple.com RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 guzzoni.apple.com A RESPONSE_ERROR 16ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 ocsp2.apple.com PASS 17ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 ocsp2.apple.com A PASS 18ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 stackpath.bootstrapcdn.com A RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 stackpath.bootstrapcdn.com PASS 16ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 redwood-labs.s3.amazonaws.com A RESPONSE_ERROR 16ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 cdnjs.cloudflare.com A RESPONSE_ERROR 16ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 fonts.gstatic.com A PASS 0ms - [2021-09-17 13:50:07] 10.49.0.2 cdnjs.cloudflare.com RESPONSE_ERROR 20ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 www.googletagmanager.com REJECT 0ms cloudflare-security [2021-09-17 13:50:07] 10.49.0.2 www.googletagmanager.com A REJECT 0ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 fonts.gstatic.com RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 cdn.jsdelivr.net A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 fonts.googleapis.com A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 fonts.googleapis.com RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 s3.us-west-1.amazonaws.com RESPONSE_ERROR 18ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 s3.us-west-1.amazonaws.com A RESPONSE_ERROR 18ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 cdn.jsdelivr.net RESPONSE_ERROR 17ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 ajax.googleapis.com A RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 ajax.googleapis.com RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 ocsp2.g.aaplimg.com A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 ocsp2.g.aaplimg.com RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 a1845.dscg2.akamai.net RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 redwood-labs.s3.amazonaws.com PASS 105ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 gstaticadssl.l.google.com RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 s3-us-west-1-w.amazonaws.com RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 s3-us-west-1-w.amazonaws.com A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 player.simplecast.com A RESPONSE_ERROR 14ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 player.simplecast.com RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 allthehacks.substack.com RESPONSE_ERROR 12ms cloudflare-security [2021-09-17 13:50:08] 10.49.0.2 allthehacks.substack.com A RESPONSE_ERROR 15ms cloudflare-security [2021-09-17 13:50:09] 10.49.0.2 podpage.imgix.net PASS 20ms cloudflare-security [2021-09-17 13:50:09] 10.49.0.2 podpage.imgix.net A PASS 43ms cloudflare-security [2021-09-17 13:50:09] 10.49.0.2 dualstack.com.imgix.map.fastly.net A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:09] 10.49.0.2 dualstack.com.imgix.map.fastly.net PASS 17ms cloudflare-security [2021-09-17 13:50:10] 10.49.0.2 client-api.itunes.apple.com A RESPONSE_ERROR 13ms cloudflare-security [2021-09-17 13:50:10] 10.49.0.2 client-api.itunes.apple.com PASS 15ms cloudflare-securityThe same request for a host name would sometimes succeed with
PASSsubsequent to aRESPONSE_ERRORfailure (see the last two lines above).I changed dnscrypt-proxy's config to use
cloudflareorcloudflare-security, and theRESPONSE_ERRORresponse codes immediately stopped and everything works again. If I enable a cloudflare name server, the intermittent errors repeat.I built a fresh Algo server this morning and was able to repeat the issue: using cloudflare name servers consistently results in
RESPONSE_ERRORresponse codes.Also reported this to the slack channel.
@QuentinMoss how did you enable logging on dnscrypt?
@mikedizon You can enable logging for dnscrypt-proxy like this:
- Set
file = '/tmp/dns.log.tmp'under[query_log]in /etc/dnscrypt-proxy/dnscrypt-proxy.toml -
sudo systemctl restart dnscrypt-proxy - tail -f /tmp/dns.log.tmp
It's weird to log to /tmp/, but that's the easiest way to do it because the system is hardened to prevent writing files to /var/log/ without adding extra permissions. Just remember to reverse these steps to disable logging so you don't fill up your /tmp/ dir.
A quick way to test a lot of DNS lookups (if you want to see if dnscrypt-proxy has lookup errors):
curl -sL http://s3.amazonaws.com/alexa-static/top-1m.csv.zip | funzip | cut -d , -f 2 | xargs -n 1 -P 10 host
This downloads Alexa's list of the top 1-million domain names, and does a dns lookup on each using parallelized host processes (change -P 10 to match your number of CPU cores).
While doing this, run tail -f /tmp/dns.log.tmp to look for errors.
Testing all 1 million domains will take about 6 hours, so you can hit control + c to stop after a few minutes once you're satisfied all is well.
I just tested again with cloudflare-security and it's still generating lots of RESPONSE_ERROR responses. For now, I'm using ['quad9-dnscrypt-ip4-filter-pri', 'cleanbrowsing-security']. 🥸