DNS New Features: `disableCache`, `finalQuery`, `unexpectedIPs`, `"*"`, `UseSystem-queryStrategy`, `useSystemHosts`

[patterniha]

New features:

disableCache for each DNS-Server-Object:

currently we have only one global disableCache option that affects all DNS-servers, but we may want to disable the cache only for a specific DNS-Server.

///

finalQuery for DNS-Server-Object:**

Suppose you want to use DNS-Server-A for "youtube.com", but use DNS-Server-B for other google sites and use DNS-Server-C for others, so you should set:

"dns":{
    "servers": [
      {
       "address": "server-A",
       "domains": ["youtube.com"]
      },
      {
       "address": "server-B",
       "domains": ["geosite:google"]
      },
      "server-C"
    ]
}

But for whatever reason, server-A may be unavailable for a while(for example, the network may be unreachable for a while) so it uses server-B for "youtube.com", but we don't want this to happen.

Currently, there is no mechanism to prevent using server-B for "youtube.com", this is due to strange behavior of skipFallback (except creating custom-geosite where "youtube.com" is removed from "google" list, but this is not possible for all users)

but now we can set finalQuery= true for server-A, so any result from server-A return as a final-result and no other DNS-server will be performed.

"dns":{
    "servers": [
      {
       "address": "server-A",
       "domains": ["youtube.com"],
       "finalQuery": true
      },
      {
       "address": "server-B",
       "domains": ["geosite:google"]
      },
      "server-C"
    ]
}

///

unexpectedIPs for DNS-Server-Object:**

Suppose we want no IP to be in an IP-range-A, and if all IPs in IP-range-A, the next-dns-fallback performed. for example for Serverless-for-Iran anti-sanction-version, i want to use a anti-sanction DNS, but goverment-run-anti-sanction-DNS only bypass sanctions and not filter. IRGFW return 10.10.34.0/24, 2001:4188:2:600:10:10:34:0/120 range for blocked domain, so if the return-IPs is in these ranges, the fallback-DNS should be performed.

one way to achieve this goal is creating custom-geosite and then using ! sign, but this is not possible for all users. another way is to calculate reverse-CIDR-list, for example using online-tools to calculate reverse-CIDR-list, but the reverse-CIDR-list is long and it causes the configuration to be messy.

as a result i add unexpectedIPs option, and an IP is matched if and only if does not match any of the IP-ranges in the unexpectedIPs list, in other words: expectedIPs = [0.0.0.0/0, ::/0] - unexpectedIPs.

also, we may need all IPs to be in range-A, and no IP to be in range B, so we need both expectedIPS and unexpectedIPs: expectedIPS=[range-A], unexpectedIPs=[range-B]

"dns":{
    "servers": [
      {
       "address": "anti-sanction-dns",
       "unexpectedIPs": ["10.10.34.0/24", "2001:4188:2:600:10:10:34:0/120"]
      },
      "fallback-dns"
    ]
}

///

Add to Documentation:

(dns-server settings)

disableCache: bool

* The cache is disabled, if disableCache = true in DNS-settings or this DNS-Server-settings.

///

finalQuery：bool

* when true, the result is returned in any case(even when IP-list is empty) and no other fallback will be performed.

///

expectedIPs: [string]

...

if you add "*" in this list, the original-IPs still returned if no ip matched.


///
unexpectedIPs: [string]

* reverse form of expectedIPs, an IP is matched if and only if does not match any of the IP-ranges in the list, in other words: 
`expectedIPs = [0.0.0.0/0, ::/0] - unexpectedIPs.`

if you add "*" in this list, the original-IPs still returned if no ip matched.

///
queryStrategy: "UseIP" | "UseIPv4" | "UseIPv6" | "UseSystem"

when queryStrategy is UseSystem, every time dns-Query call, it check the system-network to see if it supports IPv6(and IPv4) or not, if it support IPv6(or IPv4), the IPv6(or IPv4) is also returned, otherwise not returned.

///
(dns settings)

useSystemHosts: bool

if true, system-hosts appends to config-hosts at start, default is false.

[j2rong4cn]

@RPRX 我写了个只返回rcode的DNS服务，要不要合并到这里？

{
  "domains": [
    "geosite:category-ads-all",
    "domain:lan",
    "regexp:^[^\\.]*$"
  ],
  "skipFallback": true,
  "finalQuery": true,
  "address": "rcode://3"
}

https://github.com/j2rong4cn/Xray-core/commit/d08f64833eecc6c843a6831f2593f0207bf5a646

[patterniha]

@j2rong4cn

although, it is possible to add rcode-only-dns-server but it is better to add this feature to the DNS-hosts instead of adding rcode-only-dns-server.

i change the hosts code to achieve this goal, but currently we only have rcode=0: https://github.com/XTLS/Xray-core/pull/4673

there is no difference between rcode=0, and rcode=3 or others in practice, because we don't return any IP in any case.

anyway, if rcode-number is important for your job, you can extend hosts code to return your desired rcode.

anyway, @RPRX has to make the final decision and I just said my opinion.

[j2rong4cn]

@patterniha I need to return rcode=3. I used nslookup to test and found that nslookup will fallback when rcode=0. I see StaticHosts.Lookup does not return an error, so I wrote rcode-only-dns-server

[patterniha]

@patterniha I need to return rcode=3. I used nslookup to test and found that nslookup will fallback when rcode=0. I see StaticHosts.Lookup does not return an error, so I wrote rcode-only-dns-server

so i think it is better to extend hosts code, for example:

"geosite:category-ads-all": "#3"

This is definitely a better and more direct way, anyway @RPRX gives the final opinion.

[j2rong4cn]

@patterniha I need to return rcode=3. I used nslookup to test and found that nslookup will fallback when rcode=0. I see StaticHosts.Lookup does not return an error, so I wrote rcode-only-dns-server

so i think it is better to extend hosts code, for example:

"geosite:category-ads-all": "#3"

This is definitely a better and more direct way, anyway @RPRX gives the final opinion.

Good, let's do as you say.

[RPRX]

简单看了一下，直接把 geoip 改为 expected_geoip 这种 breaking change 是不行的，会严重损坏现有配置

[patterniha]

After a quick look, it is not feasible to geoipchange directly to this breaking change, which will seriously damage the existing configuration.expected_geoip

@RPRX

No, it is 100% compatible with current configuration.

it does not change any current logic and configuration.

I just move codes to "condition_geoip.go", because we need to have more general function for unexpectedIPs, priorIPs and similar needs in the future.

It seems like you looked too quickly.

///

The only break is allowUnexpectedIPs and I don't think it has been used and it can be safely changed to priorIPs.

[gfw-killer]

Thank you, as you are focused on Xray DNS, please have a look at this Issue too https://github.com/XTLS/Xray-core/issues/4677

[j2rong4cn]

so i think it is better to extend hosts code, for example:

"geosite:category-ads-all": "#3"

Check this out https://github.com/j2rong4cn/Xray-core/commit/43b71df77f1aea140031034e78e6b1aa4875171b

[patterniha]

so i think it is better to extend hosts code, for example: "geosite:category-ads-all": "#3"

Check this out j2rong4cn@43b71df

you should open a PR, I'm just a contributor like you.

[RPRX]

我觉得用 priorIPs 取代 allowUnexpectedIPs 是合理的，不过 geo 是支持反选的比如 "geoip:!cn"，所以两个 un 似乎没必要

[RPRX]

不过现在反选只支持 geo，如果有需要的话再加一个反选 IP/CIDR 的语法就差不多了

[RPRX]

还有就是先 expectedIPs 再 priorIPs，看名称的话顺序不太对，应当把 priorIPs 改名为 secondExpectedIPs

[patterniha]

I think it is reasonable to priorIPsreplace with allowUnexpectedIPs, but geo supports inverse selection "geoip:!cn", so two uns seem unnecessary.

@RPRX

Yes, support inverse but:

1. You cannot and two inverse, for example you cannot have !geoip:cloudflare && !geoip:cloudfront
2. You can inverse only geoip, and you cannot inverse Non-geoip cird, for example i need inverse of {10.10.34.0/24 + 2001:4188:2:600:10:10:34:0/120} for serverless-for-Iran-anti-saction but we can't even have inverse of {10.10.34.0/24}, let alone the inverse of both.

[patterniha]

Also, first , expectedIPsthen priorIPs, the order is not right if you look at the names, you should priorIPschange tosecondExpectedIPs

First, We will rarely need both at the same time. Second, Even if we have both, it doesn't matter whether we apply expectIPs first or priorIPs first.

[RPRX]

First, We will rarely need both at the same time. Second, Even if we have both, it doesn't matter whether we apply expectIPs first or priorIPs first.

这俩不就是为了配合起来使用的吗，expect 两次，且有一个明确在前面，~~还有人要的话就 third~~

[RPRX]

话说我突然想到，expectedIPs 作为一个数组，可能也像 https://github.com/XTLS/Xray-core/pull/4673 以前的 hosts 一样把内容给合并了

或许直接更改它的行为，改为不合并，确保数组内第一个就是 first，第二个就是 second，以此类推，~~还是说本来就是这样的逻辑~~

[patterniha]

ExpectedIPs, unexpectedIPs, priorIPs, unpriorIPs Are 4 distinct options, Although we can have several of them at the same time,but it is a rarely used.

ExpectedIPs is clear.

unexpectedIPs is reverse-form of ExpectedIPs and i explained why we need reverse-form too.

priorIPs logic is simple: act same as expectedIPs but if no IP matched, the original-IPs still returned without any change, it is useful for workers/MitM configs.

unpriorIPs is reverse-form of priorIPs

[RPRX]

我误解了你，你也误解了我，我本来以为你那个 priorIPs 是填 "0.0.0.0/0" 和 "::/0" 用的，作为第二层 expected

但是这个误解正好导致一个更简洁而强大的想法：把 expectedIPs 改为逐级匹配的，这一点你还没看懂 https://github.com/XTLS/Xray-core/pull/4666#issuecomment-2843772399

至于如何合并反选，我还得再想想

[patterniha]

It suddenly occurred to me that expectedIPsas an array, it might be possible to merge the contents like before #4673hosts

Maybe just change its behavior to not merge, making sure the first one in the array is first, the second one is second, and so on.~Or is this the logic?~

Suppose you want your IPs to be in range A but not in range B. So you simply set: ExpectedIPs = [A], unexpectedIPs = [B]

Therefore, the presence of unexpectedIPs next to expectedIPs is mandatory and the goal cannot be achieved by just changing syntax.

[RPRX]

我觉得四个选项叠一起的话逻辑有点混乱，压成两个选项就行，unexpectedIPs 是可以加的，它们是合并而不是逐级，且作用于 expectedIPs 每一级匹配成功之后（除非 "0.0.0.0/0 或 "::/0"），比如第一级匹配到了但被 unexpectedIPs 否决了就继续下一级

[RPRX]

这样就可以 cover 所有需求了，更少的配置项，更强大的功能

[patterniha]

As i understand, you agree with unexpectedIPs but not with priorIPs and unpriorIPs.

So what is alternative for priorIPs = [geoip:cloudflare] in your method?

[RPRX]

比如如下配置

"expectedIPs": ["geoip:us", "geoip:uk", "0.0.0.0/0", "::/0"],
"unexpectedIPs": ["10.10.34.0/24", "2001:4188:2:600:10:10:34:0/120"]

逻辑是

1. expect US IPs, if we picked an IP, check all unexpectedIPs, if no IP was picked or the picked IP is "unexpected" then
2. expect UK IPs, if we picked an IP, check all unexpectedIPs, if no IP was picked or the picked IP is "unexpected" then
3. allow all IPv4, do not check unexpectedIPs
4. allow all IPv6, do not check unexpectedIPs

[patterniha]

1. This break current expectedIPs behavior.

   If the IPs contain both uk and us, now we only have us ips.

2. Many changes need to be made in the code.

3. It is definitely more complicated than using 4 options

[RPRX]

这并不会严重破坏 expectedIPs 的行为，只是把 US 放第一级、UK 放第二级了，这可以接受，用户配置里本来就是这个顺序，我不确定现有的代码逻辑里是合并了还是逐级匹配，但是改成逐级匹配没有难度，就只是改成 for 循环，比多两个选项要简单

并且如果有人要 third 或者更多，他也可以通过优先级顺序来实现

[RPRX]

简单来说就是你的 PR 在广义范围上的 expected 功能只有两级且第二级只能全选，我提的是 expected 多级且每级范围可以自定义

[patterniha]

Let's do this:

I remove priorIPs and unpriorIPs options and just add unexpectedIPs.

Because we can use redirect-socks-in/out So these options are not necessary and are just confusing for users.

and don't change the current expectedIPs behavior

Ok?

[RPRX]

and don't change the current `expectedIPs" behavior

这个可以 change 啊，有什么问题吗，可以实现把更细分的 IP 范围放前面以优先匹配，比如 CF、US、ALL

[patterniha]

It is 5 am in Iran now. I don't think there is a problem with your solution, but I have to think about it.