Add support for nftables set

[mm2231031]

Prerequisites

• [X] I have checked the Wiki and Discussions and found no answer

• [X] I have searched other issues and found no duplicates

• [X] I want to request a feature or enhancement and not ask a question

Description

What problem are you trying to solve?

Now that nftables is widely used, some systems no longer support the ipset function. It is reasonable to add support for nftset now.

Proposed solution

Dnsmasq v2.87 has added support for nfset.

Alternatives considered

Additional information

[Voidstranger]

Agreed, very useful feature .

[cjom]

Now that nftables is widely used, some systems no longer support the ipset function. It is reasonable to add support for nftset now.

OpenWrt no longer supports ipset, now is nft sets.

[emlimap]

There is a go module to work with nftables. It is still in early stages as per readme https://github.com/google/nftables

[hayzamjs]

+1

[lexbritvin]

Maybe it would be better to implement some hooks for CRUD operations. In yaml, we may list:

hooks:
  add:
    - domains: [domains as in ipset or *]
       command: ./script-add.sh

The scripts will be called with os.Exec like script.sh domain ip It also covers webhooks #3662 as users may implement their own curl scripts. There can also be predefined hooks for ipset and nftables for less tech-savy people.

The proposal is more prone to errors, but we won't rely on ipset or nftables directly. Run errors may be ignored by the app but with a warning log. That way, we won't wait for user specific feature requests for operations related to AGH internal processes.

What do you think?

[cjom]

Very interesting idea, I did not know about hooks.

[Ovear]

It sounds promising, but I am curious about the running time cost.

Additionally, I don't think these two serve the same purpose. Webhook usually occur after a query, and don't disturb the query itself, whereas IPset/NFset we are talking there need to be executed exactly before the query results are sent to clients.

[lexbritvin]

Very good point.

We may define timeouts. Let's say for critical applications like ipset/nfset we will have blocking timeout of 200ms (or any other reasonable time for dns resolution). If blocking timeout exceeds, we just continue execution ignoring the ipset. We may also give users a field to configure this timeout if they are running some slow hardware. They will know why DNS is slower than expected. For cases like webhooks, the deadline for the script to run in the background is 10s-30s (also configurable) not to block the whole app.

Again, as I suggested for ipset and nfset we may have predefined implementations. Such implementations may not have timeouts as they know what they do internally. Having an internal tool for writing hooks may increase feature adoption like this FR for nftables.

Yaml example:

hooks:
  pre-add:
    - domains: [domains as in ipset or *]
      type: ipset|nfset|... # predefined implementations
  post-add:
    - domains: [domains as in ipset or *]
      command: ./script-add.sh
      timeout: 200ms # Time until we wait for the script to finish successfully. May be -1 to block, 0 no wait, >0 for timeout.
      deadline: 30s # Time until we kill the script

[Ovear]

Hi,

That's a great solution. However, there is another aspect we should consider.

When compareing the post/pre time to the time the query result is sent to the client, different scenarios may arise. For performance consideration, it would be beneficial to add an option specify whether the hook should be called when the reply hit the cache/TTL. This is also may an optimization measure to what is applied in IPset/NFset implementations.

[this-username-has-been-taken]

I second that! Would be great to fill nft sets directly from AdGuard.

So far I came with a workaround:

1. Installed a dnsmasq.
2. Defined nft sets there.
3. Installed an AdGuardHome.
4. Made AdGuardHome listen an alternative port.
5. Set dnsmasq cache size to zero and pointed it to AdGuardHome as the upstream server.

It works, but:

1. Since https://github.com/AdguardTeam/AdGuardHome/issues/1727 is not implemented yet the only client I see in the AdGuardHome is dnsmasq server. All other clients are hidden behind it even though I set add-subnet option for the dnsmasq.
2. "Paired" DNS servers work slower than one.

[abdullahtahiriyo]

@this-username-has-been-taken

Thanks a lot for your setup.

I need an nft set to block a domain in the firewall (fw4, openwrt 23.05.5), inspired by your setup, I ended up modifying it by:

• AdGuardHome listens at port 53
• Openwrt DNSMasq listens at port 54
• Openwrt DNSMasq has a proper configuration to create and populate the set (ipset tab in DHCP and DNS configuration)
• AdGuardHome has a domain specific upstream (in addition to the real upstreams), redirecting to DNSMasq: [/domaintobeblockedinfirewall.com/]127.0.0.1:54

This solves the problem of reducing speed of DNS resolution via AGH, when coming from OpenwrtDNSMasq due to forking. The disadvantage is that you need to set the to be blocked domain in two places, one in the Openwrt DNSMasq configuration, and another one in the upstream configuration.

[this-username-has-been-taken]

@abdullahtahiriyo , That is another way to do it. Thanks! You might also try to edit dnsmasq_ipset_add() method in the /etc/init.d/dnsmasq script so that when a new host is added to the IP Set it will be also added into the AdGuard config automatically.

Also I didn't noticed much of a speed drop of the DNS resolution while using dnsmasq (main) -> AdGuardHome (upstream) setup.

[njhsi]

https://gist.github.com/njhsi/a91c43427b8306af25a1a1a7f4877bc9

an initial support to nftset, same format to dnsmasq, noted that "inet" table is now hard coded, of openwrt...

[quyenvsp]

Hi, please anyway to resolve this? I current upgraded to OpenWrt 24.10.0 then pbr-iptables stopped work. It now required nftables which AGH not supported yet.

[serj95reg]

Hi, any news?

[YBS-PC]

Please add support for nftables set