Preview of records if zone doesn't exist yet

[Koopzington]

Could we make preview display the records that would be created even if the zone doesn't exist yet? Right now (at least for the Hetzner DNS Provider if that's relevant) i only get to see the created records when i push and let dnscontrol automatically create the zone. It would be nice if we could review them in the preview output first though.

[tlimoncelli]

Hi there!

I'm having difficulty reproducing this. Can you send me sample output?

Also, please include the output of dnscontrol version

Thanks!

[Koopzington]

var REG_NONE = NewRegistrar('none')
var DSP_HETZNER = NewDnsProvider('hetzner')

D('example.org', REG_NONE, DnsProvider(DSP_HETZNER),
    A('@', '127.0.0.1')
)

Same result with a freshly pulled 4.12.0

[tlimoncelli]

Would you please try again with the ppreview command? I have a hunch this bug may not appear in the new code.

(ppreview is the future replacement for preview. it is a rewrite from scratch with many new features)

[Koopzington]

[tlimoncelli]

CC @das7pad

[das7pad]

On the phone, I can check in detail when I'm back late next week.

@Koopzington can you try again with the latest version of dnscontrol and a different, random domain name?

[tlimoncelli]

I just tested this with the BIND provider and I get a full preview. I wanted to verify that this isn't a global problem.

I look forward to @Koopzington 's results. I'm optimistic.

[das7pad]

I can reproduce the report with a random domain. It looks like something is not quite right in dnscontrol.

var REG_NONE = NewRegistrar('none')
var DSP_HETZNER = NewDnsProvider('HETZNER')

D('testing-2024-12-27.dev', REG_NONE, DnsProvider(DSP_HETZNER),
    A('@', '127.0.0.1')
)

HEAD (6d5bfe952041979bd915830df18c317c25368e72)

$ dnscontrol preview
CONCURRENTLY gathering 1 zone(s)
SERIALLY gathering 0 zone(s)
Waiting for concurrent gathering(s) to complete...DONE
******************** Domain: testing-2024-12-27.dev
#1: Ensuring zone "testing-2024-12-27.dev" exists in "HETZNER"
INFO#1: Domain "testing-2024-12-27.dev" provider HETZNER Error: "testing-2024-12-27.dev" is not a zone in this HETZNER account
INFO#1: DetermineNS: zone "testing-2024-12-27.dev"; Error: error while getting Nameservers for zone="testing-2024-12-27.dev" with provider="HETZNER": "testing-2024-12-27.dev" is not a zone in this HETZNER account
Done. 0 corrections.

With extra logging:

diff --git a/providers/hetzner/hetznerProvider.go b/providers/hetzner/hetznerProvider.go
index 414b2ecf..832bd351 100644
--- a/providers/hetzner/hetznerProvider.go
+++ b/providers/hetzner/hetznerProvider.go
@@ -59,2 +59,3 @@ func New(settings map[string]string, _ json.RawMessage) (providers.DNSServicePro
 func (api *hetznerProvider) EnsureZoneExists(domain string) error {
+       fmt.Println("\nHETZNER -> EnsureZoneExists", domain)
        domains, err := api.ListZones()
@@ -70,2 +71,3 @@ func (api *hetznerProvider) EnsureZoneExists(domain string) error {
 
+       fmt.Println("\nHETZNER -> EnsureZoneExists", domain, "createZone")
        if err = api.createZone(domain); err != nil {
@@ -145,2 +147,3 @@ func (api *hetznerProvider) GetZoneRecordsCorrections(dc *models.DomainConfig, e
 func (api *hetznerProvider) GetNameservers(domain string) ([]*models.Nameserver, error) {
+       fmt.Println("\nHETZNER -> GetNameservers", domain)
        z, err := api.getZone(domain)
@@ -155,2 +158,3 @@ func (api *hetznerProvider) GetNameservers(domain string) ([]*models.Nameserver,
 func (api *hetznerProvider) GetZoneRecords(domain string, meta map[string]string) (models.Records, error) {
+       fmt.Println("\nHETZNER -> GetZoneRecords", domain)
        records, err := api.getAllRecords(domain)
@@ -171,2 +175,3 @@ func (api *hetznerProvider) GetZoneRecords(domain string, meta map[string]string
 func (api *hetznerProvider) ListZones() ([]string, error) {
+       fmt.Println("\nHETZNER -> ListZones")
        zones, err := api.getAllZones()

CONCURRENTLY gathering 1 zone(s)
SERIALLY gathering 0 zone(s)
Waiting for concurrent gathering(s) to complete...
HETZNER -> GetNameservers testing-2024-12-27.dev

HETZNER -> ListZones

HETZNER -> GetZoneRecords testing-2024-12-27.dev
DONE
******************** Domain: testing-2024-12-27.dev
#1: Ensuring zone "testing-2024-12-27.dev" exists in "HETZNER"
INFO#1: Domain "testing-2024-12-27.dev" provider HETZNER Error: "testing-2024-12-27.dev" is not a zone in this HETZNER account
INFO#1: DetermineNS: zone "testing-2024-12-27.dev"; Error: error while getting Nameservers for zone="testing-2024-12-27.dev" with provider="HETZNER": "testing-2024-12-27.dev" is not a zone in this HETZNER account
Done. 0 corrections.

Here is roughly what's happening: The main routine runs on the zone via oneZone:

1. discover nameservers, oneZone calls generateDelegationCorrections, which in turn calls driver.GetNameservers with the domain, which in turn fails as the nameservers are zone bound and the zone does not exist yet -> this logs INFO#1: DetermineNS: zone "testing-2024-12-27.dev"; Error: error while getting Nameservers for zone="testing-2024-12-27.dev" with provider="HETZNER": "testing-2024-12-27.dev" is not a zone in this HETZNER account
2. check zone exists, oneZone calls generatePopulateCorrections per provider, which in turn calls driver.ListZones, which in turn succeeds but does not yield the requested zone as it does not exist yet -> this logs #1: Ensuring zone "testing-2024-12-27.dev" exists in "HETZNER"
3. discover zone records, oneZone calls generateZoneCorrections per provider, which in turn calls driver.GetZoneRecords with the domain, which in turn fails as the zone does not exist yet -> this logs INFO#1: Domain "testing-2024-12-27.dev" provider HETZNER Error: "testing-2024-12-27.dev" is not a zone in this HETZNER account

TL;DR: dnscontrol preview tries to discover nameservers and records for a non-existing zone. It does not execute EnsureZoneExists.

On a glance, this seems to be affecting all providers that need to "create" a resource before they can list zone records or discover nameservers. This looks like a bug in dnscontrol. To be able to list nameservers and zone records, EnsureZoneExists needs to run, not just get scheduled, as part of dnscontrol preview already.

Proposed fix:

I think it would be sensible to change the order of operations in oneZone to check for missing zones (step 2 above) per provider ahead of listing zone resources (nameservers in step 1; and zone records step 3).

Auto-creating zones on preview could be a follow-up change (i.e. run EnsureZoneExists on the spot).

[Koopzington]

Sorry for the delayed response but yeah i can confirm i'm getting the same output as das7pad gets with 4.12.5.

[tlimoncelli]

@das7pad: I agree with your analysis and your proposal. Would you like to make a PR?

[das7pad]

@das7pad: I agree with your analysis and your proposal. Would you like to make a PR?

Yes, I've started to look into it. The caching of zones is not quite right in a few providers (creating zones does not always invalidate the cache [in a thread-safe manner]), which I will address in advance. I will need help with testing the provider changes. Is it OK to ask the respective maintainers on the PR for help?

[tlimoncelli]

Yes, please create individual PRs. Thanks for taking the initiative to get this fixed!

[das7pad]

https://github.com/StackExchange/dnscontrol/pull/3337 is available for testing.

See above for ten PRs with provider fixes/improvements to get the single pass push to work. I've mentioned the maintainer on each of them to aid with testing.