for-mac
for-mac copied to clipboard
docker
Broken TCP packets after version 4.22.0
Description
I am using wg-easy successfully inside Docker for Mac version 4.22.0. However newer versions of Docker for Mac break the TCP layer making wg-easy not working at all. I have tried the newer Docker for Mac version on both Intel and M3 MacBook with the same result.
Note: To install
tcpdumpandethtoolinsidewg-easycontainer runapk update && apk add tcpdump ethtool.
Docker for Mac 4.22.0 (working)
client - another macbook
$ curl -vvv cheat.sh
<web page content>
$ ifconfig utun4
utun4: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1420
options=6460<TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
inet 10.0.4.2 --> 10.0.4.2 netmask 0xffffff00
$ sudo tcpdump -i utun4 -v | grep cheat
tcpdump: listening on utun4, link-type RAW (Raw IP), snapshot length 524288 bytes
...
10.0.4.2.53066 > cheat.sh.http: Flags [S], cksum 0x52bf (correct), seq 3148994044, win 65535, options [mss 1380,nop,wscale 6,nop,nop,TS val 3063072233 ecr 0,sackOK,eol], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [S.], cksum 0x1fd9 (correct), seq 4170317243, ack 3148994045, win 65504, options [mss 65495,nop,nop,TS val 1634596043 ecr 3063072233,nop,wscale 5], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x3da0 (correct), ack 1, win 2052, options [nop,nop,TS val 3063072236 ecr 1634596043], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [P.], cksum 0x99f9 (correct), seq 1:72, ack 1, win 2052, options [nop,nop,TS val 3063072236 ecr 1634596043], length 71: HTTP, length: 71
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x3d5f (correct), ack 72, win 2044, options [nop,nop,TS val 1634596045 ecr 3063072236], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x29bd (correct), seq 1:1369, ack 72, win 16384, options [nop,nop,TS val 1634596122 ecr 3063072236], length 1368: HTTP, length: 1368
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xa974 (correct), seq 1369:2737, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x3778 (correct), ack 1369, win 2030, options [nop,nop,TS val 3063072316 ecr 1634596122], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x3234 (correct), ack 2737, win 2009, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x390c (correct), seq 2737:4105, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x5a0b (correct), seq 4105:5473, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x2788 (correct), ack 5473, win 2005, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x691b (correct), seq 5473:6841, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x2246 (correct), ack 6841, win 1983, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x2205 (correct), ack 6841, win 2048, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x8f40 (correct), seq 6841:8209, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x1cc3 (correct), ack 8209, win 2026, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xad47 (correct), seq 8209:9577, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x176b (correct), ack 9577, win 2026, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xc67f (correct), seq 9577:10945, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x1228 (correct), ack 10945, win 2005, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x11fd (correct), ack 10945, win 2048, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x3fc5 (correct), seq 10945:12313, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x0cba (correct), ack 12313, win 2026, options [nop,nop,TS val 3063072317 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x6947 (correct), seq 12313:13681, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x0762 (correct), ack 13681, win 2026, options [nop,nop,TS val 3063072317 ecr 1634596123], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [P.], cksum 0xd6b4 (correct), seq 13681:14481, ack 72, win 16384, options [nop,nop,TS val 1634596127 ecr 3063072316], length 800: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0x0432 (correct), ack 14481, win 2035, options [nop,nop,TS val 3063072320 ecr 1634596127], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x78ad (correct), seq 14481:15849, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xfed3 (correct), ack 15849, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xc5ab (correct), seq 15849:17217, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xf97b (correct), ack 17217, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xacc9 (correct), seq 17217:18585, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xf423 (correct), ack 18585, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x4b8c (correct), seq 18585:19953, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xeecb (correct), ack 19953, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x3190 (correct), seq 19953:21321, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xe973 (correct), ack 21321, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xd516 (correct), seq 21321:22689, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0xc318 (correct), seq 22689:24057, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 1368: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xe405 (correct), ack 22689, win 2048, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [P.], cksum 0xeb37 (correct), seq 24057:24617, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 560: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xdc9c (correct), ack 24617, win 2017, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [P.], cksum 0x0630 (correct), seq 24617:25785, ack 72, win 16384, options [nop,nop,TS val 1634596139 ecr 3063072327], length 1168: HTTP
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xd7f7 (correct), ack 25785, win 2029, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [F.], cksum 0x9fe8 (correct), seq 25785, ack 72, win 16384, options [nop,nop,TS val 1634596139 ecr 3063072327], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [.], cksum 0xd7e3 (correct), ack 25786, win 2048, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
10.0.4.2.53066 > cheat.sh.http: Flags [F.], cksum 0xd7e2 (correct), seq 72, ack 25786, win 2048, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
cheat.sh.http > 10.0.4.2.53066: Flags [.], cksum 0x9fe0 (correct), ack 73, win 16383, options [nop,nop,TS val 1634596142 ecr 3063072332], length 0
server - wg-easy container
# ifconfig wg0
wg0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.0.4.1 P-t-P:10.0.4.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MTU:1420 Metric:1
RX packets:166629 errors:4118 dropped:0 overruns:0 frame:4118
TX packets:461695 errors:0 dropped:1518 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:65040452 (62.0 MiB) TX bytes:535859376 (511.0 MiB)
# tcpdump -i eth0 -v | grep cheat
tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
...
83957e66aad1.53066 > cheat.sh.80: Flags [S], cksum 0x51bf (correct), seq 3148994044, win 65535, options [mss 1380,nop,wscale 6,nop,nop,TS val 3063072233 ecr 0,sackOK,eol], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [S.], cksum 0x1ed9 (correct), seq 4170317243, ack 3148994045, win 65504, options [mss 65495,nop,nop,TS val 1634596043 ecr 3063072233,nop,wscale 5], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x3ca0 (correct), ack 1, win 2052, options [nop,nop,TS val 3063072236 ecr 1634596043], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [P.], cksum 0x98f9 (correct), seq 1:72, ack 1, win 2052, options [nop,nop,TS val 3063072236 ecr 1634596043], length 71: HTTP, length: 71
cheat.sh.80 > 83957e66aad1.53066: Flags [.], cksum 0x3c5f (correct), ack 72, win 2044, options [nop,nop,TS val 1634596045 ecr 3063072236], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [.], cksum 0x28bd (correct), seq 1:1369, ack 72, win 16384, options [nop,nop,TS val 1634596122 ecr 3063072236], length 1368: HTTP, length: 1368
cheat.sh.80 > 83957e66aad1.53066: Flags [.], cksum 0x3806 (incorrect -> 0x1375), seq 1369:13681, ack 72, win 16384, options [nop,nop,TS val 1634596123 ecr 3063072236], length 12312: HTTP
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x3678 (correct), ack 1369, win 2030, options [nop,nop,TS val 3063072316 ecr 1634596122], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x3134 (correct), ack 2737, win 2009, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x2688 (correct), ack 5473, win 2005, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x2146 (correct), ack 6841, win 1983, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x2105 (correct), ack 6841, win 2048, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [P.], cksum 0xd5b4 (correct), seq 13681:14481, ack 72, win 16384, options [nop,nop,TS val 1634596127 ecr 3063072316], length 800: HTTP
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x1bc3 (correct), ack 8209, win 2026, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x166b (correct), ack 9577, win 2026, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x1128 (correct), ack 10945, win 2005, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x10fd (correct), ack 10945, win 2048, options [nop,nop,TS val 3063072316 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x0bba (correct), ack 12313, win 2026, options [nop,nop,TS val 3063072317 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x0662 (correct), ack 13681, win 2026, options [nop,nop,TS val 3063072317 ecr 1634596123], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0x0332 (correct), ack 14481, win 2035, options [nop,nop,TS val 3063072320 ecr 1634596127], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [.], cksum 0x129e (incorrect -> 0x7694), seq 14481:17217, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 2736: HTTP
cheat.sh.80 > 83957e66aad1.53066: Flags [P.], cksum 0x24d6 (incorrect -> 0x3556), seq 17217:24617, ack 72, win 16384, options [nop,nop,TS val 1634596135 ecr 3063072320], length 7400: HTTP
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xfdd3 (correct), ack 15849, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xf87b (correct), ack 17217, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xf323 (correct), ack 18585, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xedcb (correct), ack 19953, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xe873 (correct), ack 21321, win 2026, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xe305 (correct), ack 22689, win 2048, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xdb9c (correct), ack 24617, win 2017, options [nop,nop,TS val 3063072327 ecr 1634596135], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [P.], cksum 0x0530 (correct), seq 24617:25785, ack 72, win 16384, options [nop,nop,TS val 1634596139 ecr 3063072327], length 1168: HTTP
cheat.sh.80 > 83957e66aad1.53066: Flags [F.], cksum 0x9ee8 (correct), seq 25785, ack 72, win 16384, options [nop,nop,TS val 1634596139 ecr 3063072327], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xd6f7 (correct), ack 25785, win 2029, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [.], cksum 0xd6e3 (correct), ack 25786, win 2048, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
83957e66aad1.53066 > cheat.sh.80: Flags [F.], cksum 0xd6e2 (correct), seq 72, ack 25786, win 2048, options [nop,nop,TS val 3063072332 ecr 1634596139], length 0
cheat.sh.80 > 83957e66aad1.53066: Flags [.], cksum 0x9ee0 (correct), ack 73, win 16383, options [nop,nop,TS val 1634596142 ecr 3063072332], length 0
TCP is successfully established in the beginning - S -> S. -> . (3-way handshake, see documentation). Then follows the web page data.
Docker for Mac 4.32.0 (broken)
client - another macbook
$ curl -vvv cheat.sh
* Host cheat.sh:80 was resolved.
* IPv6: (none)
* IPv4: 5.9.243.188
* Trying 5.9.243.188:80...
* connect to 5.9.243.188 port 80 from 10.0.4.2 port 55544 failed: Operation timed out
* Failed to connect to cheat.sh port 80 after 75011 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to cheat.sh port 80 after 75011 ms: Couldn't connect to server
$ ifconfig utun4
utun4: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1420
options=6460<TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
inet 10.0.4.2 --> 10.0.4.2 netmask 0xffffff00
$ sudo tcpdump -i utun4 -v | grep cheat
tcpdump: listening on utun4, link-type RAW (Raw IP), snapshot length 524288 bytes
...
10.0.4.2.52544 > cheat.sh.http: Flags [SEW], cksum 0x3e35 (correct), seq 2571259145, win 65535, options [mss 1380,nop,wscale 6,nop,nop,TS val 1537466895 ecr 0,sackOK,eol], length 0
cheat.sh.http > 10.0.4.2.52544: Flags [S.], cksum 0xf3a9 (incorrect -> 0x43d3), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.http > 10.0.4.2.52544: Flags [S.], cksum 0xf3a9 (incorrect -> 0x43d3), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.http > 10.0.4.2.52544: Flags [S.], cksum 0xf3a9 (incorrect -> 0x43d3), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.http > 10.0.4.2.52544: Flags [S.], cksum 0xf3a9 (incorrect -> 0x43d3), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.http > 10.0.4.2.52315: Flags [S.], cksum 0xf3a9 (incorrect -> 0x09bc), seq 4203615612, ack 1431375901, win 65408, options [mss 65495,nop,nop,TS val 1650844888 ecr 4173577889,nop,wscale 7], length 0
cheat.sh.http > 10.0.4.2.52544: Flags [S.], cksum 0xf3a9 (incorrect -> 0x43d3), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
server - wg-easy container
# ifconfig wg0
wg0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.0.4.1 P-t-P:10.0.4.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MTU:65455 Metric:1
RX packets:5490 errors:1582 dropped:0 overruns:0 frame:1582
TX packets:6953 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:858572 (838.4 KiB) TX bytes:1053276 (1.0 MiB)
# tcpdump -i eth0 -v | grep cheat
tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
...
706a40b241c9.52544 > cheat.sh.80: Flags [SEW], cksum 0xa023 (correct), seq 2571259145, win 65535, options [mss 1380,nop,wscale 6,nop,nop,TS val 1537466895 ecr 0,sackOK,eol], length 0
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
192.168.65.7.53 > 706a40b241c9.46412: 52544 1/0/0 188.243.9.5.in-addr.arpa. PTR cheat.sh. (88)
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.80 > 706a40b241c9.52315: Flags [S.], cksum 0x5598 (incorrect -> 0x6baa), seq 4203615612, ack 1431375901, win 65408, options [mss 65495,nop,nop,TS val 1650844888 ecr 4173577889,nop,wscale 7], length 0
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
cheat.sh.80 > 706a40b241c9.52544: Flags [S.], cksum 0x5598 (incorrect -> 0xa5c1), seq 2401073651, ack 2571259146, win 65408, options [mss 65495,nop,nop,TS val 1650895273 ecr 1537466895,nop,wscale 7], length 0
There seems to be problem with the S. packet delivery to curl from website. curl then can not complete the 3-way handshake and times out after a while.
I have read that the cksum incorrect can be false positive due to computation offloading to NIC, but it does seem like it is the cause.
What I have tried
Inspect NAT and routing
Before tcpdump I was looking at iptables -vxn -L and iptables -vxn -L -t nat for some routing issues. But the tcpdump shows that the packets are routed properly all the way to the curl.
Inspect checksum computation
I have compared output of ethtool -k wg0 and ethtool -k eth0 from inside the wg-easy container for both Docker for Mac versions and they are the same.
I have tried to disable checksum on macbook client
$ sudo sysctl -w net.link.generic.system.hwcksum_tx=0
net.link.generic.system.hwcksum_tx: 1 -> 0
$ sudo sysctl -w net.link.generic.system.hwcksum_rx=0
net.link.generic.system.hwcksum_rx: 1 -> 0
Direct communication using nc
I can successfully comunicate between client and wg-easy container using nc
client
$ nc -l 9999
Hello
wg-easy container
# echo Hello | nc 10.0.4.2 9999
Both TCP and UDP (-u) is working.
Docker setting Use kernel networking for UDP
I have tried the switch Resources > Network > Use kernel networking for UDP in both ON and OFF positions (with restart).
Versions
Client
MacBook M1
Sonoma 14.5
WireGuard app from App Store, version 1.0.16 (27)
Server
MacBook Intel
Sonoma 14.4.1
WireGuard app from App Store, version 1.0.16 (27)
Docker for Mac 4.22.0 / 4.32.0
MacBook M3
Sonoma 14.5
WireGuard app from App Store, version 1.0.16 (27)
Docker for Mac 4.32.0
Reproduce
- Install recent version of Docker for Mac
- Configure and run
wg-easycontainer (see below) - Add VPN client in
wg-easyweb administration athttp://0.0.0.0:51821 - Install Wireguard client on another device (I have tested iPhone and another macbook)
- Add configuration via QR code (iPhone) or via download + import (macbook)
- Turn on VPN tunnel
wg-easy start command
See documentation
docker run -d \
--name=wg-easy \
-e WG_HOST=my-host.local \
-e WG_PORT=51820 \
-e WG_DEFAULT_ADDRESS=10.0.4.x \
-e WG_DEFAULT_DNS=<redacted> \
-e PASSWORD=<redacted> \
-v ./data:/etc/wireguard \
-p 0.0.0.0:51820:51820/udp \
-p 0.0.0.0:51821:51821/tcp \
--cap-add=NET_ADMIN \
--cap-add=SYS_MODULE \
--sysctl="net.ipv4.conf.all.src_valid_mark=1" \
--sysctl="net.ipv4.ip_forward=1" \
--restart always \
ghcr.io/wg-easy/wg-easy:13
wg0.conf (generated after startup and when adding new clients, at ./data/wg0.conf)
# Note: Do not edit this file directly.
# Your changes will be overwritten!
# Server
[Interface]
PrivateKey = <redacted>
Address = 10.0.4.1/24
ListenPort = 51820
PreUp =
PostUp = iptables -t nat -A POSTROUTING -s 10.0.4.0/24 -o eth0 -j MASQUERADE; iptables -A INPUT -p udp -m udp --dport 51820 -j ACCEPT; iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT;
PreDown =
PostDown = iptables -t nat -D POSTROUTING -s 10.0.4.0/24 -o eth0 -j MASQUERADE; iptables -D INPUT -p udp -m udp --dport 51820 -j ACCEPT; iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT;
# Client: MacBook M1
[Peer]
PublicKey = <redacted>
PresharedKey = <redacted>
AllowedIPs = 10.0.4.2/32
Client configuration (QR code / downloaded via web administration)
[Interface]
PrivateKey = <redacted>
Address = 10.0.4.2/24
DNS = <redacted>
[Peer]
PublicKey = <redacted>
PresharedKey = <redacted>
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = my-host.local
PersistentKeepalive = 0
Expected behavior
I expect the network to work (eg. internet web pages are loaded), being routed properly through the VPN tunnel.
docker version
Client:
Version: 27.0.3
API version: 1.46
Go version: go1.21.11
Git commit: 7d4bcd8
Built: Fri Jun 28 23:59:41 2024
OS/Arch: darwin/arm64
Context: desktop-linux
Server: Docker Desktop 4.32.0 (157355)
Engine:
Version: 27.0.3
API version: 1.46 (minimum version 1.24)
Go version: go1.21.11
Git commit: 662f78c
Built: Sat Jun 29 00:02:44 2024
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.7.18
GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc:
Version: 1.7.18
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 27.0.3
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.15.1-desktop.1
Path: /Users/ujezdsky/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.28.1-desktop.1
Path: /Users/ujezdsky/.docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.32
Path: /Users/ujezdsky/.docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.14
Path: /Users/ujezdsky/.docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /Users/ujezdsky/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /Users/ujezdsky/.docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /Users/ujezdsky/.docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.10.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-scout
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 5
Server Version: 27.0.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.6.32-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 12
Total Memory: 19.51GiB
Name: docker-desktop
ID: a121339e-8e8e-4752-a854-d387acf1c3fc
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///Users/ujezdsky/Library/Containers/com.docker.docker/Data/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
Diagnostics ID
257AADD6-96D6-41D9-87E7-605D420CD0E8/20240718105011
Additional Info
Information for older and working Docker for Mac installation (on Intel MacBook, I have not found version 4.22.0 for Apple Silicon).
Diagnostics ID
6FB77661-8FDA-427A-926F-CF4FC04C2C6A/20240718105247
docker version
Client:
Cloud integration: v1.0.35-desktop+001
Version: 24.0.5
API version: 1.43
Go version: go1.20.6
Git commit: ced0996
Built: Fri Jul 21 20:32:30 2023
OS/Arch: darwin/amd64
Context: desktop-linux
Server: Docker Desktop 4.22.0 (117440)
Engine:
Version: 24.0.5
API version: 1.43 (minimum version 1.12)
Go version: go1.20.6
Git commit: a61e2b4
Built: Fri Jul 21 20:35:45 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.21
GitCommit: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc:
Version: 1.1.7
GitCommit: v1.1.7-0-g860f061
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 24.0.5
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.1
Path: /Users/ujezdsky/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.20.2-desktop.1
Path: /Users/ujezdsky/.docker/cli-plugins/docker-compose
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: /Users/ujezdsky/.docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.6
Path: /Users/ujezdsky/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-sbom
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-scan
scout: Command line tool for Docker Scout (Docker Inc.)
Version: 0.20.0
Path: /Users/ujezdsky/.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/ujezdsky/.docker/cli-plugins/docker-debug" is not valid: failed to fetch metadata: fork/exec /Users/ujezdsky/.docker/cli-plugins/docker-debug: no such file or directory
WARNING: Plugin "/Users/ujezdsky/.docker/cli-plugins/docker-feedback" is not valid: failed to fetch metadata: fork/exec /Users/ujezdsky/.docker/cli-plugins/docker-feedback: no such file or directory
Server:
Containers: 3
Running: 2
Paused: 0
Stopped: 1
Images: 91
Server Version: 24.0.5
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc version: v1.1.7-0-g860f061
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 5.15.49-linuxkit-pr
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 19.55GiB
Name: docker-desktop
ID: c83038e4-bf16-4723-8add-55b750b74864
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
