XTLS
feature: happy eyeballs (RFC 8305) for UseIP-domainStrategy
(test on: windows-v25.3.3)
in freedom-outbound and a tcp-connection, or in proxy-outbound and a tcp/udp(quic)-connection, suppose that our connect-address is a "domain" and [ip1, ip2, ip3, ip4] is the list of resolved-IPs.
Current behavior:
- it tries to connect to ip1 for 30 seconds, and if it fails, it tries to connect to ip2 for 30 seconds, and ...
- so if, for example, ip1 and ip2 are unavailable for any reason, It takes 60 seconds to connect to the ip3 and as a result, it takes 60 seconds to connect to the destination !!!
Correct behavior (happy eyeballs RFC 8305):
- First, we sort the IP-list in such a way that v4 and v6 IPs appear one after the other, for example if ip1 and ip2 are version-4, and ip3 and ip4 are version-6 after sorting we have [ip1, ip3, ip2, ip4].
- then try to connect to ip1 in thread1, after for example 300ms try to connect to ip3 in thread2, ...
- As a result several attempts may be in flight at the same time, and whenever one of the attempts succeed, all other connections are cancelled, and the winning connection is used. This means a much shorter wait before one of the IP addresses connect successfully.
First application:
- Happy Eyeballs is particularly important for dual-stack clients, when some hosts may have resolvable IPv6 addresses that are somehow unreachable.
Second application:
- For any reason, some IPs may not be available, as a result, with happy eyeballs, we can find the IP that can be connected as soon as possible.
Application in iran:
- In Iran some instagram IPs are blocked and some are not, in resolved-ip-list some IPs are blocked and some are not, as a result, with happy eyeballs, we can find the IP that can be connected as soon as possible.
tips:
- For Happy eyballs, it doesn't matter whether the IP is version 4 or 6, but it is better to sort the resolved-ip-list so that v4 and v6 IPs appear one after the other.(RFC 8305)
- happy eyeballs is used in freedom-outbound when address is a domain and connection is tcp,
- or is used in proxy-outbounds when address is domain and connection is tcp or udp(quic).
/// in python stock asyncio supports Happy Eyeballs, so we just need to set the time to wait for a connection attempt to complete, before starting the next attempt in parallel.
so there should be a ready library for happy eyeballs for tcp-connections in Go too.
freedom在直出没设置domainstrategy的情况下是有happy eyeball的(golang自带) 如果设置了domainstrategy那没办法 因为代码里的架构只设计了传递单个ip 改函数签名得死一堆下游
@Fangliding even, with domainStrategy = "AsIs" happy eyeballs does not perform, the only change will be made is that instead of 30 seconds, it tries for 5 seconds for each IP.
Although, "AsIs" is not useful for my purpose, because instagram domains are blocked in iran.
/// in python i had this problem too, i solved this problem by hijacking default resolver (by changing the build-in codes)
so in "Go" we can change the build-in-Go-codes to use our IP-list instead of query to the system.
@Fangliding Even if you don't plan on adding this feature, at least change the 30 seconds to 5 seconds, like golang.
this is better than nothing, because for instagram domains we have only two IPs, and one of which is usually blocked. so in the worst case scenario, it takes 5.3 seconds to connect instead of 30.3
想了想还是不行。UseIP是从列表里随机抽，即使我把超时时间改成1秒，也需要耗费几秒才能建立一个连接，这是完全不合适的。这个超时尝试的功能只适合从一个列表里筛掉少部分不可用的IP而不是期望它高效从一堆不可用IP里筛出可用的，即使你认为这样勉强能用也不适合为此再单开一个field，你完全可以去用asis
First, as I said, Happy eyeballs doesn't even work for AsIs.(or maybe works with 5 seconds! while rfc recommends 250ms)
Second, if happy eyeballs was only for special purposes, we wouldn't have an RFC about it. even its 3rd version has been published recently, and it is currently in the draft https://datatracker.ietf.org/doc/draft-pauly-happy-happyeyeballs-v3/
Third, we don't need to rewrite happy eyeballs, just change the default resolver to use our ip-list instead of query to the system, if domainStrategy is UseIP/ForceIP.
First, as I said, Happy eyeballs doesn't even work for AsIs.(or maybe works with 5 seconds! while rfc recommends 250ms)
250毫秒是传出第二个尝试队列的延迟 不是超时时间 你连它的含义是什么都不知道
Second, if happy eyeballs was only for special purposes, we wouldn't have an RFC about it. even its 3rd version has been published recently, and it is currently in the draft https://datatracker.ietf.org/doc/draft-pauly-happy-happyeyeballs-v3/
Third, we don't need to rewrite happy eyeballs, just change the default resolver to use our ip-list instead of query to the system, if domainStrategy is UseIP/ForceIP.
Happy eyeballs 的主要作用还是决定使用ipv4还是v6, 不是从一堆可能不可靠的IP中挑出一个好用的。asis的情况下域名被丢到golang自带的dialer里进行处理，这是golang处理的，它实现了一个简单的v4v6的racer，这也只是个建议性的RFC，没必要完全听它的
我前面已经说过了整套系统本来只设计了处理单个目标 resolver也不那么好hook 这是个大项目不是你专门为了这个目的写的python小玩具，没有可比性，你这点用例不值当改这么多，要不你自己开个fork玩吧，这是我最后一次解释了，以后我在看到类似issue我就直接关了
250 milliseconds is the delay for sending the second attempt queue, not the timeout. You don't even know what it means.
I know this, you again ...
@RPRX
Why didn't you care about serverless methods? we are not in china. happy eyeballs is a basic feature that every basic network app should have. (although it is useful even in proxy-outbound)
Without happy eyeballs and reality-noise, Serverless-for-iran is still incomplete.
///
I was really hoping that with your help we can rid Iran of vpn sellers.
But you only answer them, when they ask about xhttp. They only care about their customers and money.
Currently, we do not need methods like xhttp in Iran,
but everyone in the group is involved with xhttp.
///
Xray-core has advanced methods such as reality, xhttp,... But it doesn't have a simple feature like Happy eyeballs, This is completely like a joke.
///
It seems like you are working for vpn sellers instead of working for freedom
250 milliseconds is the delay for sending the second attempt queue, not the timeout. You don't even know what it means.
I know this, you again ...
绝大多数happy eyeballs的实现比如go都是分v4v6两个队列请求，你举的ins没有ipv6 所以它只有一个v4队列 所以这个delay不会发挥作用 就是一个队列后面等前面超时
Why didn't you care about serverless methods? we are not in china. happy eyeballs is a basic feature that every basic network app should have. (although it is useful even in proxy-outbound)
Without happy eyeballs and reality-noise, Serverless-for-iran is still incomplete.
我说了实现这玩意麻烦，你只用开口就行了，实现和后续维护都和你没关系，你但凡试试开个Pr?
I was really hoping that with your help we can rid Iran of vpn sellers.
But you only answer them, when they ask about xhttp. They only care about their customers and money.
如果你说的是昨天晚上，纯粹是因为他们@我我才会去看一眼，平时我是压根不点开这个群，而且别人提的是BUG(虽然是无效反馈)，BUG是一直会处理的
Currently, we do not need methods like xhttp in Iran,
but everyone in the group is involved with xhttp.
///
Xray-core has advanced methods such as reality, xhttp,... But it doesn't have a simple feature like Happy eyeballs, This is completely like a joke.
///
It seems like you are working for vpn sellers instead of working for freedom
核心的主要目的一直是代理请求，你这个永远是次要的，如果你觉得处理XHTTP之类的传输的BUG是 working for vpn sellers 那我也没什么好说了
Most implementations of happy eyeballs, such as go, use two queues for v4 and v6 requests.
this is not happy eyeballs, it is just "Dual stack simultaneous connecting"
you just need to read the RFC 8305 abstract:
Happy Eyeballs Version 2: Better Connectivity Using Concurrency
Abstract
Many communication protocols operating over the modern Internet use
hostnames. These often resolve to multiple IP addresses, each of
which may have different performance and connectivity
characteristics. Since specific addresses or address families (IPv4
or IPv6) may be blocked, broken, or sub-optimal on a network, clients
that attempt multiple connections in parallel have a chance of
establishing a connection more quickly. This document specifies
requirements for algorithms that reduce this user-visible delay and
provides an example algorithm, referred to as "Happy Eyeballs".
...
///
in python happy eyeballs works even if we have ipv4 only, Exactly according to RFC.
The RFC only suggests that if you have both ipv4 and ipv6, instead of trying to connect to ipv4 first, then ipv6 (or vice versa) it is better to sort the list so that v4 and v6 IPs appear one after the other.
So we may have as many parallel-attempts as there are IPs in the list.
But for what you say, we have at most two parallel-attempts, and this is not happy eyeballs.
///
As an Iranian proverb says: "کس نخارد پشت من جز ناخن انگشت من"
It seems the only way is to study the structure of Xray-core and do PR myself.
Most implementations of happy eyeballs, such as go, use two queues for v4 and v6 requests.
this is not happy eyeballs, it is just "Dual stack simultaneous connecting"
you just need to read the RFC 8305 abstract:
Happy Eyeballs Version 2: Better Connectivity Using Concurrency Abstract Many communication protocols operating over the modern Internet use hostnames. These often resolve to multiple IP addresses, each of which may have different performance and connectivity characteristics. Since specific addresses or address families (IPv4 or IPv6) may be blocked, broken, or sub-optimal on a network, clients that attempt multiple connections in parallel have a chance of establishing a connection more quickly. This document specifies requirements for algorithms that reduce this user-visible delay and provides an example algorithm, referred to as "Happy Eyeballs". ...///
in python happy eyeballs works even if we have ipv4 only, Exactly according to RFC.
The RFC only suggests that if you have both ipv4 and ipv6, instead of trying to connect to ipv4 first, then ipv6 (or vice versa) it is better to sort the list so that v4 and v6 IPs appear one after the other.
So we may have as many parallel-attempts as there are IPs in the list.
But for what you say, we have at most two parallel-attempts, and this is not happy eyeballs.
///
As an Iranian proverb says: "کس نخارد پشت من جز ناخن انگشت من"
It seems the only way is to study the structure of Xray-core and do PR myself.
golang 实现了一个happy eyeballs 只是没按你想的那样 很难理解吗 哪怕是xray要实现也是这种两个队列加delay 又不是必须按你想的干
Golang implements a happy eyeballs, but it doesn't work the way you think.
This is not the way I think, this is the way the RFC thinks: Sorting Addresses
I hope I have time, and study xray-core structure and golang so I can help more.
the one that @Fangliding says (the current implementation of happy eyeballs in golang) is RFC 6555. RFC 6555 is obsoleted by RFC 8305 (the one I say and the current implementation of happy eyeballs in python).
in RFC 6555 we have two queue one for ipv4 and one for ipv6 and we have at most two parallel-attempts, so if we have ipv4 only, there will be no parallel-attempts.
but in RFC 8305 we have one ordered-list of both ipv4 and ipv6, in the ordered-list, v4 and v6 IPs should appear one after the other (or two, three,...), so we may have as many parallel-attempts as there are IPs in the ordered-list, and at runtime there is no difference between ipv4 and ipv6, and the IP types are only important for the ordered-list.
///
also we have happy eyeballs v3 (draft) with ECH,... added to it.
所以呢 这东西始终都是建议性的 标准库都没实现的8305你还 "happy eyeballs is a basic feature that every basic network app should have." 要不你去给golang开个pr吧 反正挂在这一样没人写
So this thing is always a suggestion and the standard library has not implemented it. 8305 You still say "happy eyeballs is a basic feature that every basic network app should have." Or you can open a PR for golang. It's just hanging here and no one has written it.
I don't care about RFC, I have a need for serverless-for-Iran and this need is exactly met with RFC 8305,
However, apart from this, RFC 8305 must be implemented in Xray-core because it creates a better experience for any type of application (my need is only one of them)
@Fangliding
Also, golang-PR doesn't help at all. Because we also need happy eyeballs for quic/h3(only make sense for proxy-outbounds)
But golang built-in happy eyeballs, is only for tcp.
So, we have to write our own happy eyeballs.
这么说吧 核心现在两个(以后还可能加)解析地址的方法本质不是指定域名的IP list，而是按域名重置目标地址为IP，和happy eyeaballs根本不兼容，只有全保持域名请求的情况下会被送到golang的dialer会被它内部进行处理，要干也是golang的net包干，xray层面没有合适的方法去操作golang dialer内部的逻辑，linkname已经被禁止了。 还挂着的一个issue甚至还考虑在其他位置把域解析为IP，如果又在哪强行实现happy eyeballs只会让这套逻辑更加混乱，这不是clash系感不感兴趣，就没代理软件自己搓这个的，你希望的是解析出10个IP就一起开10个连接，这在你的Python小程序的可行，但是对应xray来说是不可接受的，这根本不叫happy eyeballs。自己实现甚至会导致出现请求有特征(独特的并发重试行为) . 这绝对是不行的。总的来说这东西就不该xray实现，应该golang实现，golang改了xray更新后也是自动的，没必要保持这个issue 在UDP实现更是体现了你的无知，UDP根本没有连接状态 何来happy eyeballs一说？
UDP does not have a connection state at all, so how can there be happy eyeballs?
Not for raw udp, and not for freedom-outbound. For quic/h3 and for proxy-outbounds.
那也是出站自己的racer 和快乐眼球没有关系 更不是"we have to write our own happy eyeballs" 的理由 再说了99%的情况下都是一台服务端，压根没有竞速的理由，撑死就是h2/h3竞速，这个反倒有pr
The implementation of happy eyeballs in python is very simple and it doesn't affect other things.
Just we have multi connect-attempts in threads(coroutines) and the first successful is selected and others ignored.
@RPRX
This issue should be open until Go implements RFC 8305 (and we use that with some changes in built-in codes for domainStrategy-UseIP)[only tcp]
Or find the golang-ready-library for RFC 8305.
Or we implement our own.
@Fangliding @RPRX
Some Instagram(meta) Ips are blocked in iran and Some not, In 99% of cases, there is at least one IP in the IP list that is not blocked.
So, we can access Instagram with using "sniffing" -> "ForceIP" ->"happy eyeballs" -> "fragment/domain-fronting".
Instagram has more than 40 million users in Iran.
Apart from this, For whatever reason, some IPs may be unavailable for a while, and happy eyeballs makes us connect to the alternate IPs as quickly as possible.
In general, this thing should not be implemented in xray, but in golang. After golang is modified, xray updates are also automatic. There is no need to keep this issue
??? We don't even have RFC 6555 for domainStrategy-UseIp/ForceIP right now. AsIs is useless, Instagram domains are blocked.
What you want is to resolve 10 IPs and open 10 connections at the same time.
we can limit the number of parallel-attempts to for example 4 ( with 5 seconds timeout for each) (and 300ms delay for each attempts)
Also, most of the time we have a maximum of 4 IPs.
Currently, we only have RFC 6555 and only for domainStrategy-AsIs and only for TCP, which is useless.
@RPRX @Fangliding @rPDmYQ
I test several cases and this is the result:
- quic-go doesn't have happy eyeballs at all.
- built-in-go-tcp only implement obsolete RFC 6555, and it can only be used for domainStrategy-AsIs.
闹麻了我还抽时间去看了chromium的代码，人家也是分v4v6部分并行，和golang一样的。更不可能管这个issue了，golang那个也是happy eyeballs v2, Sorting Addresses 是在DNS部分实现的，这我很明确因为我研究过标准库的DNS系统，之前还被你误导了。并发两条都只是实现选择而已。
@Fangliding
Happy eyeballs-v2 (RFC 8305) is clear.
Even the Go developers says what i say: https://github.com/golang/go/issues/68795#issuecomment-2305607519
Anyway, with serverless-for-iran Instagram is accessible in 50% cases, but with happy eyeballs it is accessible in 99% cases.
Instagram is the most popular social media in Iran.
Even rfc 6555 is better than nothing, but we don't have it for domainStrategy-UseIp, so it is useless.
Apart from this, For whatever reason, some IPs may be unavailable for a while, and happy eyeballs makes us connect to the alternate IPs as quickly as possible.
This is also implement for china, this is global issue.
什么global issue 总之互联网上最大的请求发起者Chrome和golang的标准库全是这个逻辑 你应该先去说服他们 虽然他们可能也只是留着一个几年不管的issue