Wireguard speeds/bandwidth very slow compared to other apps.

[yuannan]

I have a 1000Mib/s up and down connection to my ISP. My PC can do the full bandwidth without a VPN, with a VPN I can get around 800, which is very respectable.

However, on my phone Rethink Wireguard speeds are very poor. Compared to the other apps, it's less around 25% which isn't great.

	Raw	VPN
PC - Ethernet	1000	800
PC - WiFi	900	700
Phone - WiFi - Wireguard App	900	600
Phone - Wifi - WG Tunnel	900	600
Phone - WiFi - Rethink	900	150

I've disabled logging and many of the features in the App, but I've not been able to get it over 180 in the best situation. With just Rethink and no wireguard enabled I'm able to hit 900-1000, so I'm pretty sure it's the wireguard implementation that is causing this slow down. I've tried this with 2 different VPNs so I'm pretty confident it is Rethink.

Is there any settings I can try and enable/disable? Or is this likely a bug?

[ignoramous]

150mbps? That's likely more than enough for a smartphone one would think (:

Or is this likely a bug?

Not really, but the effect of running all packets through a userspace firewall. Rethink can be sped up by completely overhauling its current architecture, but I doubt the benefits will be worth it. We could probably target 200mbps or even 300mbps by tuning performance in the existing code, but that's a project for another time, especially not when there's so many open bugs and other easier to implement features that are missing.

If technical folks are reading this, I welcome pull requests and/or more investigation in to what could be improved, where & how, though.

[yuannan]

I've also noticed that ipv6 with wireguard doesn't work properly.

I'll see if I've got some time to take a dive.

[ignoramous]

@yuannan Will you please test if v055t (which is a pre-release) is any better than 150mbps: https://github.com/celzero/rethink-app/releases/tag/v0.5.5t?

[yuannan]

@yuannan Will you please test if v055t (which is a pre-release) is any better than 150mbps: https://github.com/celzero/rethink-app/releases/tag/v0.5.5t?

Thanks for reading my issue. The new apps looks great and has some cool new features but its not any faster. Or at least not noticeable. I got 160 vs the usual 150 on my first run, but I'd call that statistical fluctuations.

From what I understand, this should be a limit of the underlying firestack interface, yes? Is there a way to bench that by itself?

The wireguard and wg tunnels app is much much faster, they also let you block certain apps as well. So I presume this is also possible to speed Rethink up.

[Lanius-collaris]

@ignoramous I haven't tested, but I'm sure 4 KiB buffer is too small. If my memory serves, to take full advantage of segmentation offloads, about 64 KiB buffer is needed. https://github.com/celzero/firestack/blob/271e3d33b83ace94c6d8d97827bbae2fbaeb8b80/intra/core/cp.go#L57 https://github.com/celzero/firestack/blob/271e3d33b83ace94c6d8d97827bbae2fbaeb8b80/intra/core/buf.go#L53-L55

[ignoramous]

I haven't tested, but I'm sure 4 KiB buffer is too small.

I think we should bring back the experimental 10k MTU for the TUN device #1124 (the max allowed by Android limited I guess due to jumbo frame size?), if we can be certain that netstack's involvement (sockisification) means egress UDP (incl over IPv6) continues to work for all apps (TCP does, when we tested it).

If my memory serves, to take full advantage of segmentation offloads, about 64 KiB buffer is needed.

Currently, firestack does not implement GSO/GRO on the TUN device.

We could increase the buffer size further up from ~~4KiB~~ 16KiB if it helps UDP, but I don't think it'll improve anything for TCP as core/cp.go prefers WriterTo / ReaderFrom (which both retrier & TCPConn confirm to) before it'll exec raw copy.

[Lanius-collaris]

I haven't tested, but I'm sure 4 KiB buffer is too small. If my memory serves, to take full advantage of segmentation offloads, about 64 KiB buffer is needed.

ignoramous: We could increase the buffer size further up from ~~4KiB~~ 16KiB if it helps UDP, but I don't think it'll improve anything for TCP as core/cp.go prefers WriterTo / ReaderFrom (which both retrier & TCPConn confirm to) before it'll exec raw copy.

Although gonet.TCPConn doesn't implement WriterTo / ReaderFrom, but using 64 KiB buffer in core.Stream() doesn't help. Sigh. Will rerun test with flight recorder enabled, hope that it will give a clue.

Currently, firestack does not implement GSO/GRO on the TUN device.

IOCTL TUNSETIFF is a privileged operation, can't be used on Android. https://blog.cloudflare.com/virtual-networking-101-understanding-tap/

    struct ifreq ifr = {};
    strncpy(ifr.ifr_name, tap_name, IFNAMSIZ);
    ifr.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_VNET_HDR;
    int r = ioctl(tap_fd, TUNSETIFF, &ifr);
    if (r != 0) {
   	 error(-1, errno, "ioctl(TUNSETIFF)");
    }

( firestack may want IFF_TUN instead of IFF_TAP ) log:

type=1400 audit(0.0:1276): avc: denied { ioctl } for path="/dev/tun" dev="tmpfs" ino=435 ioctlcmd=0x54ca …

[ignoramous]

IOCTL TUNSETIFF is a privileged operation, can't be used on Android.

Thanks. That's a bummer. Is multi-queue TUN allowed?

Will rerun test with flight recorder

Not sure it captures much as there's limited memory assigned to it by firestack, right now. We could increase it, if needed.

---

got 160 vs the usual 150 on my first run, but I'd call that statistical fluctuations

@yuannan v055u is out now. If you use Rethink's GitHub/Website variant, will you check how much bandwidth improvement, if any, you see? Optionally, turn ON Configure -> Network -> Bandwidth booster and see if uploads improve? Note that, Bandwidth booster is an experimental feature.

On my Android, bandwidth is shaped by an on-device policy, apparently. Apps I use seemingly are not allowed to cross 150mbps down & 250mbps up.

[yuannan]

@ignoramous I tried the new version. I see no major difference if anything it is lower?? I think it's probably just run to run variance for me.

• PC Raw: 950

• PC Wireguard: 700

• Raw: 800

• Wireguard : 500

• v055u: 80-120

• v055u booster: 80-110

I tried using firefox and fast.com as well as using speedtest-go in termux. Both times the results were inconclusive/not improved.

Is there anything I can provide for you guys that can help solve this? It seems like v055u has worked for you better 🤔

[ignoramous]

Bummer.

What's the MTU of WireGuard configuration you're testing in Rethink? Set it to empty (which switches it to (auto)) and test then speed test with Bandwidth booster also turned ON?

Other than that, we may have to overhaul the way WireGuard works today to get more bandwidth out of it, which is not an easy task & won't be done for months (but a task we intend to take up soon-ish).

[yuannan]

What's the MTU of WireGuard configuration you're testing in Rethink?

They've always been on auto.

Other than that, we may have to overhaul the way WireGuard works today to get more bandwidth out of it, which is not an easy task & won't be done for months (but a task we intend to take up soon-ish).

Is there anything I can do to help? I've had a look thru the Rethink Stack and it seems fairly complicated. I usually write C++ so I don't think I would be much help directly :/

[ignoramous]

I've had a look thru the Rethink Stack and it seems fairly complicated.

Yeah, but you may have good help with any popular code-understanding LLM.

Is there anything I can do to help?

If you want to learn Go, you can submit a patch to bypass Netstack for wgproxy.go's Dial() methods, entirely. This won't be easy, nor straight-forward. It is likely an extensive change. I have a draft of it in my laptop, but I was at it on-and-off for 2 weeks and the refactor was still not done, and now it stands abandoned since we want to finish shipping v055u / v055v / v055w within this year.

[yuannan]

I've had a look thru the Rethink Stack and it seems fairly complicated.

Yeah, but you may have good help with any popular code-understanding LLM.

Is there anything I can do to help?

If you want to learn Go, you can submit a patch to bypass Netstack for wgproxy.go's Dial() methods, entirely. This won't be easy, nor straight-forward. It is likely an extensive change. I have a draft of it in my laptop, but I was at it on-and-off for 2 weeks and the refactor was still not done, and now it stands abandoned since we want to finish shipping v055u / v055v / v055w within this year.

I've written Go before and I'll see if I can contribute in anyway. My HPC work in C++ tends to be quite good but idk if that will translate over meaningfully.