Unit tests: improve idempotence of generating data files

[marc-vanderwal]

Currently, the way that data files are generated with ZONEMASTER_RECORD=1 make test or Zonemaster::Engine::Nameserver::save() has a few shortcomings that make it difficult to:

• actually see what changed: git diff shows a huge mess of deletions and insertions that completely hide any meaningful changes;
• sort out merge conflicts when several people made changes to said files.

This is due to two main reasons:

Firstly, the cache is implemented as a hash of hashes of hashes and Zonemaster::Engine::Nameserver::save() just iterates over the keys of the level 1 and 2 hashes without sorting them beforehand. This means that the cache entries are written out in random order, essentially rewriting the whole file. I propose that these hashes be walked through using a sorted list of keys (i.e. with for my $k (sort keys %h) instead of for my $k (keys %h)).

Secondly, packets are written as a binary representation of their wire formats, including the ID field. Because that field is randomized, two semantically identical packets can still differ only by their IDs. As a result, we are saving a different packet whenever we call Zonemaster::Engine::Nameserver::save(), even if nothing else changed. Besides, the packet’s transaction ID doesn’t matter anymore at the level of abstraction of Zonemaster::Engine::Nameserver. I therefore propose that the ID field of the packet’s header be set to zero when caching a response (similar to what is done in DNS over HTTPS to improve cache-friendliness; see RFC 8484, § 4.1).

The two changes I propose are non-breaking and don’t even require regeneration of the .data files. Existing files can be sorted using Unix’s sort command.

Implementing both of these changes shouldn’t break anything at all, yet will ensure that running ZONEMASTER_RECORD=1 make test twice in a row will result in semantically identical .data files where changes are easier to spot with git diff.

[tgreenx]

Partly relates to #938