Add support for special use domains

[ruihildt]

It would be nice to be able to parse special use domains and recognize them as such. Right now it's not the case.

Maybe with a isSpecialUse boolean?

See: https://datatracker.ietf.org/doc/html/rfc6761

[remusao]

Hi @ruihildt, I am not very familiar with special use domains. Could you share some examples of expected behavior?

Thanks,

[ruihildt]

I'm working on a webextension which is dynamically proxying based on URL, and right now if I use isPrivate and isIcann, most of these domains are not recognized.

I have actually tried to understand if they should be falling under isPrivate, but didn't find relevant documentation.

I'll extract the relevant part of the spec for each special domain below. It's unclear to me whether tldts would fall under Application software or Name resolution APIs and libraries (or if this is even relevant).

In any case, I think it would be nice to be able to parse these when using tldts.

---

6.1. Domain Name Reservation Considerations for Private Addresses

The private-address [RFC1918] reverse-mapping domains listed below, and any names falling within those domains, are Special-Use Domain Names:

 **10.in-addr.arpa.      21.172.in-addr.arpa.  26.172.in-addr.arpa.
 16.172.in-addr.arpa.  22.172.in-addr.arpa.  27.172.in-addr.arpa.
 17.172.in-addr.arpa.  30.172.in-addr.arpa.  28.172.in-addr.arpa.
 18.172.in-addr.arpa.  23.172.in-addr.arpa.  29.172.in-addr.arpa.
 19.172.in-addr.arpa.  24.172.in-addr.arpa.  31.172.in-addr.arpa.
 20.172.in-addr.arpa.  25.172.in-addr.arpa.  168.192.in-addr.arpa.**

These domains, and any names falling within these domains, are special in the following ways:

1. Users are free to use these names as they would any other reverse-mapping names. However, since there is no central authority responsible for use of private addresses, users SHOULD be aware that these names are likely to yield different results on different networks.

2. Application software SHOULD NOT recognize these names as special, and SHOULD use these names as they would other reverse-mapping names.

3. Name resolution APIs and libraries SHOULD NOT recognize these names as special and SHOULD NOT treat them differently. Name resolution APIs SHOULD send queries for these names to their configured caching DNS server(s).

---

6.2. Domain Name Reservation Considerations for "test."

The domain "test.", and any names falling within ".test.", are special in the following ways:

1. Users are free to use these test names as they would any other domain names. However, since there is no central authority responsible for use of test names, users SHOULD be aware that these names are likely to yield different results on different networks.

2. Application software SHOULD NOT recognize test names as special, and SHOULD use test names as they would other domain names.

3. Name resolution APIs and libraries SHOULD NOT recognize test names as special and SHOULD NOT treat them differently. Name resolution APIs SHOULD send queries for test names to their configured caching DNS server(s).

---

6.3. Domain Name Reservation Considerations for "localhost."

The domain "localhost." and any names falling within ".localhost." are special in the following ways:

1. Users are free to use localhost names as they would any other domain names. Users may assume that IPv4 and IPv6 address queries for localhost names will always resolve to the respective IP loopback address.

2. Application software MAY recognize localhost names as special, or MAY pass them to name resolution APIs as they would for other domain names.

3. Name resolution APIs and libraries SHOULD recognize localhost names as special and SHOULD always return the IP loopback address for address queries and negative responses for all other query types. Name resolution APIs SHOULD NOT send queries for localhost names to their configured caching DNS server(s).

---

6.4. Domain Name Reservation Considerations for "invalid."

The domain "invalid." and any names falling within ".invalid." are special in the ways listed below. In the text below, the term "invalid" is used in quotes to signify such names, as opposed to names that may be invalid for other reasons (e.g., being too long).

1. Users are free to use "invalid" names as they would any other domain names. Users MAY assume that queries for "invalid" names will always return NXDOMAIN responses.

2. Application software MAY recognize "invalid" names as special or MAY pass them to name resolution APIs as they would for other domain names.

3. Name resolution APIs and libraries SHOULD recognize "invalid" names as special and SHOULD always return immediate negative responses. Name resolution APIs SHOULD NOT send queries for "invalid" names to their configured caching DNS server(s).

---

6.5. Domain Name Reservation Considerations for Example Domains

The domains "example.", "example.com.", "example.net.", "example.org.", and any names falling within those domains, are special in the following ways:

1. Users SHOULD understand that example names are reserved for use in documentation.

2. Application software SHOULD NOT recognize example names as special and SHOULD use example names as they would other domain names.

3. Name resolution APIs and libraries SHOULD NOT recognize example names as special and SHOULD NOT treat them differently. Name resolution APIs SHOULD send queries for example names to their configured caching DNS server(s).

---

While internal. is actually not part of the spec yet. Here's the working draft: https://datatracker.ietf.org/doc/html/draft-davies-internal-tld-02

[remusao]

Hi @ruihildt and sorry for not getting back to you sooner. Thanks for the documentation on these special use domains. If this issue is still relevant to you, would you be able to provide a set of test cases with current (as well as expected) tldts output on a bunch of these domains/URLs? That'd be useful. Also if you have any extra insights on what change we would need to make on tldts to handle these, it'd be great!