rec: Support for populating forward-zones using catalog zones

[kpfleming]

Short description

Adds support for catalog zones in recursor; their contents are used to populate the forward-zones list.

Checklist

I have:

• [x] read the CONTRIBUTING.md document
• [x] compiled this code
• [ ] tested this code
• [x] included documentation (including possible behaviour changes)
• [x] documented the code
• [ ] added or modified regression test(s)
• [ ] added or modified unit test(s)
• [x] checked that this code was merged to master

[kpfleming]

Indeed it needs to be possible to specify more that one catalog zone, which means this config option needs to have a plural name instead of singular (forward-zones-catalog -> forward-zones-catalogs). The double plural is a bit odd, but it's correct (one or more catalog zones which each provide one or more forward zones).

[kpfleming]

All of the suggestions have been incorporated while rewording some of the content to better reflect the ability to specify more than one catalog zone. Thanks!

[rgacogne]

Nice, I like this feature very much :)

[omoerbeek]

I intend to work on this very soon. I'm not taking the suggested documentation though, as this feature requires a more complex configuration. What I am proposing is something like this (this is thinking in progress and subject to change!)

YAML sketch

forwarding_catalog_zones:
    - name: name of catalog zone
      allow_notify: boolean, default false
      xfr:
         addresses: [] Sequence of SocketAddress
         zoneSizeHint: number
         tsig:
           name: string
           algo: string
           secret: base64string
         refresh: number
         maxReceivedMBytes: number
         localAddress: IP address
         axfrTimeout: number
      group_property: string, default forward_rec_powerdns_com
      custom_property: string, default forward.rec.powerdns.com
      defaults:
        forwarders: [] Sequence of SocketAddress
        recurse: boolean, default false
        allow_notify: boolean default false
    - ...

The top-level allow_notify tells if the catalog zone will perform a transfer on notify (checking the SOA). The xfr fields define all the parameters needed to retrieve a zone from an autoritatieve server using (A/I)XFR, compare RPZ configuration.

Then come two properties (group_property and custom_property) which define how to interpret certain records in the catalog zone.

The defaults fields define the defaults for members in the catalog zone. A catalog zone itself can override these properties by using group properties or global custom properties. The names of the group or private properties are defined by the group_property and custom_property YAML fields. The values are TXT record containing an object that defines the values for the three YAML fields in the defaults map: forwarders, recurse and allow_notify.

In the case of a group property, the key should be the string defined as group_property and the value should be a YAML object containing the values to be overwritten.

In the case of a custom property, the TXT value should be a YAML object containing the values to be overwritten.

An example of a group property for a specific member:

group.nfwxa33.zones.catalog.invalid 0 TXT "forward_rec_powerdns_com: { forwarders: ['127.0.0.:5301'], recurse: true}"

A global custom property

forward.rec.powerdns.com.ext.catalog.invalid 0 TXT "{ forwarders: ['127.0.0.:5301'], recurse: true}"

The values are processed in a hierarchical way. Values in the YAML settings are overwritten by global custom properties which are overwritten by group properties (all if applicable).

Maybe we also want a way to select only specifc members, based on a group property. E.g. we could define in the YAML settings:

groups: [GroupA, GroupB]

to only select members that have a group property equal to GroupA or GroupB.

[kpfleming]

This seems like a good configuration model to me, although I'm still a fan of putting the 'zone retrieval' parts in a top-level zone definition, the same as they would be for an RPZ or any other zone held in the recursor.

[omoerbeek]

This seems like a good configuration model to me, although I'm still a fan of putting the 'zone retrieval' parts in a top-level zone definition, the same as they would be for an RPZ or any other zone held in the recursor.

I adapted the proposed YAML.

[Habbie]

Then come two properties (group_property and custom_property) which define how to interpret certain records in the catalog zone.

Proposal: don't do custom_property until it turns out somebody really needs it. group_property doesn't look right. The name of the group property in a catalog zone is group. The context is free form, but I suggest making it a user configurable key, like poolA, poolB, etc., which then each get a YAML part that looks just like defaults.

So:

forwarding_catalog_zones:
    - name: name of catalog zone
      allow_notify: boolean, default false
      xfr:
         addresses: [] Sequence of SocketAddress
         zoneSizeHint: number
         tsig:
           name: string
           algo: string
           secret: base64string
         refresh: number
         maxReceivedMBytes: number
         localAddress: IP address
         axfrTimeout: number
      defaults:
        forwarders: [] Sequence of SocketAddress
        recurse: boolean, default false
        allow_notify: boolean default false
      groups:  # this entire section is optional
        poolA:
          forwarders: [] Sequence of SocketAddress
          recurse: boolean, default false
          allow_notify: boolean default false
        customer835:
          forwarders: ...
          ...
    - ...

[omoerbeek]

Superseded by #14759