shadowsocks-rust icon indicating copy to clipboard operation
shadowsocks-rust copied to clipboard

Transparently proxying UDP traffic with pf?

Open Orum opened this issue 1 year ago • 75 comments
trafficstars

This is a similar symptom to my earlier issue (#1473), but as the circumstances have drastically changed I thought I'd open a new issue. To summarize the changes, I am no longer running sslocal on the host (Linux/iptables) machine; it's instead being run on my FreeBSD router which uses pf.

Obviously, this requires different firewall rules to work, but there is little if any documentation I could find for transparently proxying via pf within the shadowsocks-rust project. What I've done here is largely guesswork, but this is the rule I came up with to transparently proxy both TCP & UDP to sslocal (and then on to ssserver): rdr on $int_if inet proto { tcp, udp } to !<private> -> 127.0.0.1 port 1080

sslocal is being run on the router via the following command: sslocal -b 127.0.0.1:1080 -U --protocol redir -s 192.168.x.y:8388 -m none --tcp-redir pf --udp-redir pf

...and finally, ssserver is run with -U -m none -b 192.168.x.y:8388

This "works" in the sense that both TCP & UDP traffic are being redirected to sslocal (unlike in #1473 where only TCP traffic was ever reaching sslocal), which is then sending the traffic on to sserver. However, once again, only TCP traffic is being fully proxied correctly.

UDP traffic does arrive at sslocal without any doubt, as it shows up when running sslocal with -vvv like so (timestamps removed for brevity, and the host system's address being substituted here with simply <host>):

TRACE  tokio-runtime-worker ThreadId(05) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:235: UDP server client send to 127.0.0.1:1080, control: UdpSocketControlData { client_session_id: 7043831170210357480, server_session_id: 0, packet_id: 1, user: None }, payload length 96 bytes, packet length 103 bytes
TRACE  tokio-runtime-worker ThreadId(05) shadowsocks_service::local::redir::udprelay: crates/shadowsocks-service/src/local/redir/udprelay/mod.rs:307: received UDP packet from <host>:44849, destination 127.0.0.1:1080, length 96 bytes
TRACE  tokio-runtime-worker ThreadId(05) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:433: udp relay <host>:44849 -> 127.0.0.1:1080 (proxied) with 96 bytes

This traffic is then forwarded on to ssserver, which I can verify by watching outgoing traffic on the router via tcpdump. After they arrive at ssserver though, they fail to reach the internet, so clearly something isn't configured correctly.

My best guess as to what is happening here, based on the logs, is that sslocal thinks the destination of the UDP packet is 127.0.0.1 (i.e. the address that sslocal is running on and where incoming UDP traffic on the router is redirected to), and fails to retrieve the original destination address before pf's rdr rule took effect. This wouldn't surprise me as my rdr rule is complete guesswork, so I assume I'm missing something.

If that is indeed the problem, what should the rdr rule look like to transparently proxy UDP traffic with pf? Or am I barking up the wrong tree, and is the problem due to something else entirely?

Orum avatar May 29 '24 05:05 Orum

I have never tested on FreeBSD, so this project supports FreeBSD is still "theoretically".

I have tested pf on macOS, which supports TCP perfectly just like you did on FreeBSD. But on macOS, UDP's destination must be retrieved from ioc_getstates. This API is far different from FreeBSD's definition, so I think FreeBSD doesn't use the same mechanism to get UDP packets' destination when redirected by pf.

In your last PR, you said on FreeBSD UDP's destination could be retrieved from msg_name returned by recvfrom. Is that correct? I couldn't see any document from FreeBSD talking about it.

The 127.0.0.1:1080 in your case was exactly the value of msg_name returned from recv_dest_from.

zonyitoo avatar May 29 '24 06:05 zonyitoo

In your last PR, you said on FreeBSD UDP's destination could be retrieved from msg_name returned by recvfrom. Is that correct? I couldn't see any document from FreeBSD talking about it.

That wasn't me, so you must be thinking of someone else. While I know how to write rules for pf fairly well, I've never programmed anything to interface with it, so I'm unsure of how the API is supposed to be used.

The good news is that you can access the man pages online. There's also a forum, mailing lists, and a Discord server, all of which feature some form of development/programming discussion.

I'm interested in learning as well, but I'm still learning rust, so it may be some time until I am of use.

Orum avatar May 30 '24 01:05 Orum

Glad to know that you are familiar with FreeBSD and pf. Could you find any doc / references about how to get the "original destination address" of UDP packets?

On Linux, it is quite straight forward: https://man7.org/linux/man-pages/man7/ip.7.html (IP_RECVORIGDSTADDR).

On macOS, I got it from another post: (?) Deleted.

A famous tools mitmproxy only supports TCP on FreeBSD: https://docs.mitmproxy.org/stable/howto-transparent/ .

So it is quite hard to find a reference about how FreeBSD handles UDP redirects and how to get the original destination address programmatically.

zonyitoo avatar Jun 01 '24 03:06 zonyitoo

And you could also try the ipfw: https://forums.freebsd.org/threads/retrieving-the-destination-port-in-an-ipfwed-udp-packet.37545/

zonyitoo avatar Jun 01 '24 03:06 zonyitoo

@Orum Do you still interest in making this Project working on FreeBSD? What's the current status?

zonyitoo avatar Jun 14 '24 06:06 zonyitoo

I am still very interested in making it work. However, I think the best way to do so at this point is to just read through FreeBSD's source code. As such, I'm familiarizing myself with it, but it's going to take some time as I have other obligations and projects demanding my attention, so it's on the back burner for me right now. This is probably the best, or at least, surest way to figure out how to get the address (assuming it's even possible at present) from pf.

Additionally, I'd need to learn much more about rust before I contribute anything other than information to this project, as my knowledge of it is spotty at best. But, if I do figure out the FreeBSD side of the picture, perhaps someone here can handle that side of things once they have the information. I'm also not only willing, but happy to test things to see if they work once they're implemented.

Orum avatar Jun 14 '24 06:06 Orum

https://github.com/semigodking/redsocks/issues/200 FYI, I succeeded in running transparent proxy in FreeBSD with redsocks+SOCKS5 (sorry they are completely unrelated to shadowsocks) and ipfw, but not with pf. When I tested with pf, the UDP destination address was obtained as the transparent proxy's listen address, just as @Orum mentioned. As I mentioned in the link, in OpenBSD, where pf originates from, using divert-to rule of pf worked perfectly. divert rule seems not ported to FreeBSD's pf.

ge9 avatar Jun 14 '24 07:06 ge9

FreeBSD and OpenBSD diverged in their rules syntax quite a few years back when the latter made some changes that broke older rule sets, IIRC. I've stuck with FreeBSD, which might have an equivalent of divert-to (or not), but I don't really know what it does as I never bothered to keep up with OBSD's syntax as I don't use it.

I also don't really want to switch to ipfw, or any other firewall, as I really value simplicity and clarity of pf's rules' syntax. So many other firewalls are hard to decipher, which is undesirable to say the least when it comes to security.

In any case I will try and dedicate some time to reading through and understanding the pf/bpf code in the future, but that will be some time from now.

Orum avatar Jun 14 '24 07:06 Orum

@ge9 Have you tried shadowsocks-rust with ipfw? What problem did you see?

zonyitoo avatar Jun 14 '24 07:06 zonyitoo

I tested it and it worked, but there were a problem. shadowsocks-rust uses IPv4-mapped IPv6 address to return message to IPv4 clients: https://github.com/shadowsocks/shadowsocks-rust/blob/ac67a08f87afe54e844241fb75d8bfc018a0bd03/crates/shadowsocks-service/src/local/redir/udprelay/mod.rs#L199 However, FreeBSD disables IPv4-mapped IPv6 address by default (according to https://github.com/scala-native/scala-native/issues/3630). If I set sysctl net.inet6.ip6.v6only=0, then it worked. Except for this, normal transparent proxy setting will work.

FYI, My settings are following:

  • Initial setup (the machine already had 10.0.2.15/24)
#!/bin/sh
kldload ipfw
fwcmd=ipfw
ifconfig em0 alias 10.0.2.25 netmask 0xffffff00
$fwcmd add 100 allow all from any to any via lo0
$fwcmd add 500 fwd 127.0.0.1,22222 tcp from 10.0.2.25 to any
$fwcmd add 600 fwd 127.0.0.1,22222 udp from 10.0.2.25 to any
$fwcmd add 700 allow ip from any to any
  • sslocal -b 127.0.0.1:22222 -U --protocol redir -s 192.168.1.7:1080 --tcp-redir pf --udp-redir pf -m none
    • (specifying pf seems fine)
  • ssserver -U -s 192.168.1.7:1080 -m none (on another PC)
    • note that specifying 0.0.0.0:1080 here doesn't work

ge9 avatar Jun 16 '24 06:06 ge9

There is a SUPPORT_IPV6_TRANSPARENT flag that indicates whether the current platform supports IPv6, if it is true, then shadowsocks-rust will always convert IPv4 to IPv4-mapped-IPv6 and uses only one socket for sending back.

This is always working for Linux, because dual-stack socket was enabled by default for most distribution of Linux.

Is there a way to test if dual-stack is enabled on FreeBSD? By reading sysctl net.inet6.ip6.v6only=0 should be one possible option.

zonyitoo avatar Jun 16 '24 08:06 zonyitoo

Could you please help to run the test program on FreeBSD and see if any of these syscalls returned errors? https://stackoverflow.com/questions/30184377/how-to-detect-if-dual-stack-socket-is-supported

zonyitoo avatar Jun 16 '24 08:06 zonyitoo

cc @madeye , FreeBSD doesn't support IPv4-mapped-IPv6 by default. So my previous (years ago) PR that uses IPv6 sockets for sending back UDP packets won't be able to run properly on FreeBSD.

zonyitoo avatar Jun 16 '24 08:06 zonyitoo

This is the result of the python test program.

Traceback (most recent call last):
  File "/root/test.py", line 4, in <module>
    s.connect(('::ffff:169.254.1.1', 53))
OSError: [Errno 22] Invalid argument

If I set sysctl net.inet6.ip6.v6only=0 and commented out s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1), it worked. Otherwise, it failed.

In the shadowsocks, this is the error.

2024-06-17T03:45:10.334750916+09:00  WARN udp failed to send back 68 bytes to client 10.0.2.25:44550, from target 3.132.228.249:3478 (proxied), error: Invalid argument (os error 22)

This originates from https://github.com/shadowsocks/shadowsocks-rust/blob/ac67a08f87afe54e844241fb75d8bfc018a0bd03/crates/shadowsocks-service/src/local/redir/udprelay/mod.rs#L203 .

ge9 avatar Jun 16 '24 09:06 ge9

Please test if it is Ok without settingnet.inet6.ip6.v6only=0 manually.

zonyitoo avatar Jun 16 '24 10:06 zonyitoo

I just found a better solution. Working on it.

zonyitoo avatar Jun 16 '24 11:06 zonyitoo

Tested on OpenWRT. Please help testing it if it is working correctly on FreeBSD. @ge9

zonyitoo avatar Jun 16 '24 12:06 zonyitoo

5ba8b7d worked but 765c9e5 or cd25d25 (latest) doesn't (same error).

ge9 avatar Jun 16 '24 12:06 ge9

Please run again with -v and check these debug logs:

https://github.com/shadowsocks/shadowsocks-rust/blob/cd25d25b22a3a9b1fa5805e1482826e6214f797a/crates/shadowsocks/src/net/sys/mod.rs#L143-L181

Are they all true?

zonyitoo avatar Jun 16 '24 12:06 zonyitoo

here is the log. seems all true.

2024-06-17T06:38:52.894052461+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::redir::udprelay: crates/shadowsocks-service/src/local/redir/udprelay/mod.rs:285: received UDP packet from 10.0.2.25:65158, destination 3.132.228.249:3478, length 28 bytes
2024-06-17T06:38:52.894445849+09:00 DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:123: created udp association for 10.0.2.25:65158
2024-06-17T06:38:52.894817377+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:433: udp relay 10.0.2.25:65158 -> 3.132.228.249:3478 (proxied) with 28 bytes
2024-06-17T06:38:52.894896617+09:00 TRACE  tokio-runtime-worker ThreadId(02) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(79171555479936), interests=READABLE | WRITABLE
2024-06-17T06:38:52.895050148+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:103: connected udp remote 192.168.1.8:1080 (outbound: 192.168.1.8:1080) with ConnectOpts { user_cookie: None, bind_local_addr: None, bind_interface: None, tcp: TcpSocketOpts { send_buffer_size: None, recv_buffer_size: None, nodelay: false, fastopen: false, keepalive: Some(15s), mptcp: false }, udp: UdpSocketOpts { mtu: None } }
2024-06-17T06:38:52.895129216+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:235: UDP server client send to 3.132.228.249:3478, control: UdpSocketControlData { client_session_id: 786820869274048462, server_session_id: 0, packet_id: 1, user: None }, payload length 28 bytes, packet length 35 bytes
2024-06-17T06:38:53.135364801+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:445: UDP server client receive from 3.132.228.249:3478, control: None, packet length 75 bytes, payload length 68 bytes
2024-06-17T06:38:53.136228332+09:00 TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:607: udp relay 10.0.2.25:65158 <- 3.132.228.249:3478 (proxied) received 68 bytes
2024-06-17T06:38:53.136421625+09:00 DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:155: IpStackCapability support_ipv4=true
2024-06-17T06:38:53.136798744+09:00 DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:164: IpStackCapability support_ipv6=true
2024-06-17T06:38:53.137187564+09:00 DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:175: IpStackCapability support_ipv4_mapped_ipv6=true
2024-06-17T06:38:53.13739486+09:00 TRACE  tokio-runtime-worker ThreadId(02) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(79171555480192), interests=READABLE | WRITABLE
2024-06-17T06:38:53.137685127+09:00  WARN  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:620: udp failed to send back 68 bytes to client 10.0.2.25:65158, from target 3.132.228.249:3478 (proxied), error: Invalid argument (os error 22)

ge9 avatar Jun 16 '24 12:06 ge9

Well, so FreeBSD allows bind()ing a IPv4-mapped-IPv6 address, but doesn’t allow sendmsg to it. Programs that are written in Go should also fails with the same reason.

zonyitoo avatar Jun 16 '24 12:06 zonyitoo

Call connect() in this commit. Please help verify it again.

zonyitoo avatar Jun 16 '24 13:06 zonyitoo

Hmm it seems still not working...

2024-06-16T23:01:19.392825484Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks_service::local::redir::udprelay: crates/shadowsocks-service/src/local/redir/udprelay/mod.rs:285: received UDP packet from 10.0.2.25:13870, destination 3.132.228.249:3478, length 28 bytes
2024-06-16T23:01:19.393494564Z DEBUG  tokio-runtime-worker ThreadId(03) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:123: created udp association for 10.0.2.25:13870
2024-06-16T23:01:19.393869192Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:433: udp relay 10.0.2.25:13870 -> 3.132.228.249:3478 (proxied) with 28 bytes
2024-06-16T23:01:19.394037649Z TRACE  tokio-runtime-worker ThreadId(03) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(88188202799232), interests=READABLE | WRITABLE
2024-06-16T23:01:19.394211694Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:103: connected udp remote 192.168.1.212:1080 (outbound: 192.168.1.212:1080) with ConnectOpts { user_cookie: None, bind_local_addr: None, bind_interface: None, tcp: TcpSocketOpts { send_buffer_size: None, recv_buffer_size: None, nodelay: false, fastopen: false, keepalive: Some(15s), mptcp: false }, udp: UdpSocketOpts { mtu: None } }
2024-06-16T23:01:19.394416748Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:235: UDP server client send to 3.132.228.249:3478, control: UdpSocketControlData { client_session_id: 8860848107368566370, server_session_id: 0, packet_id: 1, user: None }, payload length 28 bytes, packet length 35 bytes
2024-06-16T23:01:19.566423525Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:445: UDP server client receive from 3.132.228.249:3478, control: None, packet length 75 bytes, payload length 68 bytes
2024-06-16T23:01:19.567810852Z TRACE  tokio-runtime-worker ThreadId(03) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:607: udp relay 10.0.2.25:13870 <- 3.132.228.249:3478 (proxied) received 68 bytes
2024-06-16T23:01:19.568463729Z DEBUG  tokio-runtime-worker ThreadId(03) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:156: IpStackCapability support_ipv4=true
2024-06-16T23:01:19.568883894Z DEBUG  tokio-runtime-worker ThreadId(03) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:165: IpStackCapability support_ipv6=true
2024-06-16T23:01:19.56980524Z DEBUG  tokio-runtime-worker ThreadId(03) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:173: IpStackCapability support_ipv4_mapped_ipv6=true
2024-06-16T23:01:19.569994091Z TRACE  tokio-runtime-worker ThreadId(03) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(88188202799488), interests=READABLE | WRITABLE
2024-06-16T23:01:19.578334536Z  WARN  tokio-runtime-worker ThreadId(03) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:620: udp failed to send back 68 bytes to client 10.0.2.25:13870, from target 3.132.228.249:3478 (proxied), error: Invalid argument (os error 22)

ge9 avatar Jun 16 '24 14:06 ge9

Replace it with the method just like the one on stackoverflow. Please try again. @ge9

zonyitoo avatar Jun 16 '24 14:06 zonyitoo

Still not working...

2024-06-16T23:53:16.161455017Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::redir::udprelay: crates/shadowsocks-service/src/local/redir/udprelay/mod.rs:285: received UDP packet from 10.0.2.25:34707, destination 3.132.228.249:3478, length 28 bytes
2024-06-16T23:53:16.161934966Z DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:123: created udp association for 10.0.2.25:34707
2024-06-16T23:53:16.162531969Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:433: udp relay 10.0.2.25:34707 -> 3.132.228.249:3478 (proxied) with 28 bytes
2024-06-16T23:53:16.162692883Z TRACE  tokio-runtime-worker ThreadId(02) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(51249433133440), interests=READABLE | WRITABLE
2024-06-16T23:53:16.162828655Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:103: connected udp remote 192.168.1.212:1080 (outbound: 192.168.1.212:1080) with ConnectOpts { user_cookie: None, bind_local_addr: None, bind_interface: None, tcp: TcpSocketOpts { send_buffer_size: None, recv_buffer_size: None, nodelay: false, fastopen: false, keepalive: Some(15s), mptcp: false }, udp: UdpSocketOpts { mtu: None } }
2024-06-16T23:53:16.16302868Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:235: UDP server client send to 3.132.228.249:3478, control: UdpSocketControlData { client_session_id: 17626901403037311660, server_session_id: 0, packet_id: 1, user: None }, payload length 28 bytes, packet length 35 bytes
2024-06-16T23:53:16.356517518Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks::relay::udprelay::proxy_socket: crates/shadowsocks/src/relay/udprelay/proxy_socket.rs:445: UDP server client receive from 3.132.228.249:3478, control: None, packet length 75 bytes, payload length 68 bytes
2024-06-16T23:53:16.357477975Z TRACE  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:607: udp relay 10.0.2.25:34707 <- 3.132.228.249:3478 (proxied) received 68 bytes
2024-06-16T23:53:16.358022457Z DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:155: IpStackCapability support_ipv4=true
2024-06-16T23:53:16.358794623Z DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:164: IpStackCapability support_ipv6=true
2024-06-16T23:53:16.359158356Z DEBUG  tokio-runtime-worker ThreadId(02) shadowsocks::net::sys: crates/shadowsocks/src/net/sys/mod.rs:172: IpStackCapability support_ipv4_mapped_ipv6=true
2024-06-16T23:53:16.361218674Z TRACE  tokio-runtime-worker ThreadId(02) mio::poll: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551: registering event source with poller: token=Token(51249433133696), interests=READABLE | WRITABLE
2024-06-16T23:53:16.362913582Z  WARN  tokio-runtime-worker ThreadId(02) shadowsocks_service::local::net::udp::association: crates/shadowsocks-service/src/local/net/udp/association.rs:620: udp failed to send back 68 bytes to client 10.0.2.25:34707, from target 3.132.228.249:3478 (proxied), error: Invalid argument (os error 22)

ge9 avatar Jun 16 '24 14:06 ge9

Udp Socket bind([::ffff:3.132.228.249]:3478) was Ok, but then sendmsg([::ffff:10.0.2.25]:34707) failed?

Is that a bug in FreeBSD? :(

Further more, in the test code, sockets bind([::ffff:127.0.0.1]:0) and connect([::ffff:127.0.0.1]:53) were all working well.

For TCP socket, bind([::ffff:127.0.0.1]:0) and then listen() Ok. For client socket, bind([::ffff:127.0.0.1]:0) and then connect() to the server socket Ok.

Getting Headache.

zonyitoo avatar Jun 16 '24 14:06 zonyitoo

(edited)

By the way, I forgot to test TCP. With ipfw, --udp-redir pf is fine, but --tcp-redir pf did not work. And TCP seems not affected by this IPv4-mapped address issue (it works fine with ipfw and --tcp-redir ipfw).

ge9 avatar Jun 16 '24 15:06 ge9

UDP works with pf but TCP doesn't? That wasn't expected.

zonyitoo avatar Jun 16 '24 15:06 zonyitoo

It's about testing with ipfw, not pf.

ge9 avatar Jun 16 '24 15:06 ge9

Ok. Please also test the TCP with ipfw. Let's fix all of them together.

zonyitoo avatar Jun 16 '24 15:06 zonyitoo