[Bug Report] 配置 lan_interface 后导致局域网内部无法 ssh

[warmingking]

Checks

• [X] I have searched the existing issues
• [X] I have read the documentation
• [X] Is it your first time sumbitting an issue

Current Behavior

使用 x86 nixos 做主路由, 开启 dae 后, 和局域网内其他设备 ssh 失败, 关闭 dae 或者 dae suspend 后 ssh 正常

Expected Behavior

dae 不影响局域网 ssh

Steps to Reproduce

1. 本机局域网 ip: 192.168.8.1/24
2. dae 配置 ( 省略了 subscription 和 groups )

global {                                                                                      
    ##### Software options.                                                                   
    tproxy_port: 12345                                                                        
    tproxy_port_protect: true                                                                 
    so_mark_from_dae: 0                                                                       
    log_level: info                                                                           
    disable_waiting_network: false                                                            
    lan_interface: br0                                                                        
    wan_interface: wan                                                                        
    auto_config_kernel_parameter: true                                                        
                                                                                              
    ##### Node connectivity check.                                                            
    tcp_check_url: 'http://cp.cloudflare.com,1.1.1.1,2606:4700:4700::1111'                    
    tcp_check_http_method: HEAD                                                               
    udp_check_dns: 'dns.google.com:53,8.8.8.8,2001:4860:4860::8888'                           
    check_interval: 60s                                                                       
    check_tolerance: 50ms                                                                     
                                                                                              
    ##### Connecting options.                                                                 
    dial_mode: domain                                                                         
    allow_insecure: false                                                                     
    sniffing_timeout: 100ms                                                                   
    tls_implementation: tls                                                                   
}
dns {                                                                                         
    ipversion_prefer: 4 
    fixed_domain_ttl {} 
    upstream {
        mosdns: 'udp://127.0.0.1:53'
    }
    routing {
        request {
            fallback: mosdns
        }
    }
}
routing {
    dscp(0x4) -> direct 
    l4proto(udp) && dport(443) -> block

    # pname(NetworkManager) -> direct
    # pname(systemd-resolved, dnsmasq) -> must_direct
    # pname(mosdns) && l4proto(udp) && dport(53) -> must_direct
    pname(cfdyndns, qbittorrent-nox, syncthing) -> direct
    pname(mosdns) -> must_rules

    dip(224.0.0.0/3, 'ff00::/8') -> direct
    dip(geoip:private) -> direct
    dip(geoip:cn) -> direct
    dip(1.12.12.12) -> direct

    domain(suffix: pzany.xyz) -> direct
    domain(geosite:CN) -> direct
    # domain(geosite:biliintl) -> sp
    # domain(geosite:bahamut) -> tw

    fallback: proxy
}

3. nft ruleset

table inet firewall {
        chain forward {
                type filter hook forward priority filter; policy drop;
                ct state { established, related } accept
                ct state invalid drop
                iifname "br0" oifname "wan" accept comment "Allow trusted LAN to WAN"
                iifname "wan" oifname "br0" ct state established,related accept comment "Allow established back to LANs"
                iifname "cni0" accept
                counter packets 0 bytes 0 drop
        }

        chain input {
                type filter hook input priority filter; policy drop;
                iifname "lo" accept
                ct state { established, related } accept
                ct state invalid drop
                meta mark 0x08000000 accept
                jump traverse-from-all-subzones-to-fw-subzones-rule
                counter packets 74348 bytes 30570485 drop
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "wan" masquerade
        }

        chain prerouting {
                type nat hook prerouting priority dstnat; policy accept;
                iifname "br0" oifname "wan" accept comment "Allow trusted LAN to WAN"
                iifname "wan" oifname "br0" ct state established,related accept comment "Allow established back to LANs"
        }

        chain rule-dhcpv6 {
                ip6 saddr fe80::/10 ip6 daddr fe80::/10 udp dport 546 accept
        }

        chain rule-icmp {
                ip6 nexthdr ipv6-icmp icmpv6 type { echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert } accept
                ip protocol icmp icmp type { echo-request, router-advertisement } accept
        }

        chain rule-nixos-firewall {
                tcp dport { 22, 443, 12345 } accept
                udp dport { 53, 12345 } accept
        }
        chain traverse-from-all-subzones-to-fw-subzones-rule {
                iifname "cni0" accept
                jump traverse-from-all-zone-to-fw-zone-rule
        }

        chain traverse-from-all-zone-to-fw-zone-rule {
                tcp dport 22 accept
                jump rule-dhcpv6
                jump rule-icmp
                jump rule-nixos-firewall
        }
}

Environment

• Dae version (use dae --version): v0.5.0
• OS (e.g cat /etc/os-release): NixOS 24.05 (Uakari)
• Kernel (e.g. uname -a): 6.1.71
• Others:

Anything else?

ssh 时抓包分析

这个包是在本机上 ssh 其他机器, 在其他机器上抓的 根据输出分析, 应该是本机没有处理到 ack 包 在本机抓包分析, 也是可以看到前两次握手 ( 频繁重传 ), 看不到第三次握手的包

[dae-prow[bot]]

Thanks for opening this issue!

[mzz2017]

0.4.0有这个问题吗？

[warmingking]

0.4.0有这个问题吗？

我是新用户, 不是从 0.4.0 升级上来的 不过昨天 0.4.0 测试过也是一样的现象

[warmingking]

检查似乎是这条规则引起的,

$ ip route show table 2023
local default dev lo scope host

所有的流量都使用 lo 传输, 不知道现在还是否必要

删除后局域网内部通信恢复 ip rule delete fwmark 0x8000000/0x8000000 lookup 2023

[mzz2017]

@warmingking 你确定删除之后dae还能正常工作吗

[warmingking]

@warmingking 你确定删除之后dae还能正常工作吗

确实不行了, 这里的原理是怎样呢

[warmingking]

@mzz2017 尝试去掉 lan_ingress assign 里面的 skb->mark = TPROXY_MARK; 也可以解决 LAN 通信问题, 同时 dae 还正常工作

不删除 ip rule

[warmingking]

@mzz2017 尝试去掉 lan_ingress assign 里面的 skb->mark = TPROXY_MARK; 也可以解决 LAN 通信问题, 同时 dae 还正常工作

不删除 ip rule

抱歉, 这时候只有本机正常, lan 下其他设备外网不正常

[warmingking]

破案了, 是 net.ipv4.conf.<interface>.src_valid_mark 导致的, 应该关闭

[senkiss]

net.ipv4.conf..src_valid_mark = 0 ？