Go (generate) replacements

[cafferata]

At the moment there is still a lot of knowledge within the static documentation directory. I would love to provide more and more pages from go generate with this knowledge. A few examples:

The providers page

1. https://github.com/StackExchange/dnscontrol/pull/2083
   • ~~With a deep link to the individual provider pages.~~
   • https://github.com/StackExchange/dnscontrol/pull/2182
2. The providers with official support.
3. The providers with contributor support and their maintainers GitHub usernames.

Individual provider page For each provider page I would like to know:

1. Is it an official/community support provider?
2. What are the supported/useable features?
3. Who is the maintainer?

The language reference

1. Provide the individual function with functions parameters in accordance with the old documentation

The GitBook menu structure

1. The providers.
2. The language reference.

Custom index pages

1. Top Level Functions
2. Domain Modifiers
3. Record Modifiers

README

1. Supported DNS providers
2. Supported Domain Registrars

My original question

What I would like is a 'regex way' to replace the content within two tags with the output of go generate. As example tags I used {{generate-start-provider-matrix}} and {{generate-end-provider-matrix}}.

Example for the Markdown file providers.md:

If a feature is definitively not supported for whatever reason, we would also like a PR to clarify why it is not supported, and fill in this entire matrix.

{{generate-start-provider-matrix}}

| Provider name | Official Support | DNS Provider | Registrar | ALIAS | AUTODNSSEC | CAA | PTR | NAPTR | SOA | SRV | SSHFP | TLSA | DS | dual host | create-domains | NO_PURGE | get-zones |
| ------------- | ---------------- | ------------ | --------- | ----- | ---------- | --- | --- | ----- | --- | --- | ----- | ---- | -- | --------- | -------------- | -------- | --------- |
| `AKAMAIEDGEDNS` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
| `AUTODNS` | ❌ | ✅ | ❌ | ✅ | ❔ | ❌ | ❌ | ❔ | ❔ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |

{{generate-end-provider-matrix}}

After adding the new provider AXFRDDNS and running the go generate command.

If a feature is definitively not supported for whatever reason, we would also like a PR to clarify why it is not supported, and fill in this entire matrix.

{{generate-start-provider-matrix}}

| Provider name | Official Support | DNS Provider | Registrar | ALIAS | AUTODNSSEC | CAA | PTR | NAPTR | SOA | SRV | SSHFP | TLSA | DS | dual host | create-domains | NO_PURGE | get-zones |
| ------------- | ---------------- | ------------ | --------- | ----- | ---------- | --- | --- | ----- | --- | --- | ----- | ---- | -- | --------- | -------------- | -------- | --------- |
| `AKAMAIEDGEDNS` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
| `AUTODNS` | ❌ | ✅ | ❌ | ✅ | ❔ | ❌ | ❌ | ❔ | ❔ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
+| `AXFRDDNS` | ❌ | ✅ | ❌ | ❔ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ | ❔ | ❌ | ❌ | ❌ | ❌ |

{{generate-end-provider-matrix}}

I'm not a Go developer, so I could use some help pointing me in the right direction. 🙏 I've looked at template.ParseFiles() but it removes the original tag making regeneration impossible.

[tlimoncelli]

I wouldn't use regexes. Here's some "thinking out loud" ideas that would work better:

Simple concatenation

In this method we concatenate parts to make the file. Suppose we have foo.md which needs a table in the middle. We would put the content above the table in "top" and the content below in "bottom":

• foo.md-top
• foo.md-bottom

The "go generate" would create the file, copy the contents of foo.md-top', output the contents of the table, then copy the contents of foo.md-top'. Done.

If we use this, we should establish a standard such as: Always put the parts in a subdirectory called "parts".

Go templates

Use the go template system. Suppose you want to generate foo.md. There would be a file called foo.md.template.

A rough sketch would be:

foo.md.template looks like:

This is the top!
{{.MyTable}}
This is the bottom!

The code looks something like:

tmpl := template.Must(template.ParseFiles("foo.md.template"))
tmpl.Execute(w, map[string]interface{}{"MyTable": variableContainingTheTable}

(something like that. I'm doing this from memory) The point is that you pass a map[string]string with the items that can be substituted into the template.

In addition to "MyTable", you could construct other bits of text that are to be injected into the template. You could even pass a value that is a list of all the providers, and use the other templating features to loop through the providers.

Start and end markers

Rather than use a regex, require that the start/end markers have to be the only thing on the line. This makes them easier to parse.

Suppose you want to rewrite the files in place with {{generate-start-provider-matrix}} and {{generate-end-provider-matrix}}, you probably want a function like this:

buf, err := ioutil.ReadFile("filename.md")
(handle error)
txt := string(buf)

newtxt := InsertInFile(txt, "provider-matrix", MyTable)

InsertInFile would split the text on newlines to make it easy to work with.

To make the new file, copy the lines up to and including {{generate-start-provider-matrix}}, then copy the MyTable data. Now skip all lines until {{generate-end-provider-matrix}}, then copy the remaining to the output.

Thoughts...

I think I like the template idea the best.

Tom

[cafferata]

Thanks for your comment/thoughts. 🙏 Before creating this GitHub issue, I had a working Go templates setup set up. The main problem I see here is that all content is duplicated in both the Go template and the generated Markdown file. As a result, almost every Markdown page has a Go templates file next to it. This does not seem to me to be a desirable situation.

With the 'Simple concatenation' solution, the content is also duplicated. What should someone make an adjustment to? In foo.md-top? Or in foo.md? These are steps that need to be described. This also does not seem to me to be a desirable situation.

[j-f1]

I’ve had success using the “start and end markers” approach before, with HTML comments as the markers, e.g.:

<!-- provider-matrix-start -->
<!-- THIS TABLE IS AUTOMATICALLY GENERATED by the code in foo/bar/baz.go. DO NOT EDIT. Run `go generate` to update. -->
| Provider name | Official Support | DNS Provider | Registrar | ALIAS | AUTODNSSEC | CAA | PTR | NAPTR | SOA | SRV | SSHFP | TLSA | DS | dual host | create-domains | NO_PURGE | get-zones |
| ------------- | ---------------- | ------------ | --------- | ----- | ---------- | --- | --- | ----- | --- | --- | ----- | ---- | -- | --------- | -------------- | -------- | --------- |
| `AKAMAIEDGEDNS` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
| `AUTODNS` | ❌ | ✅ | ❌ | ✅ | ❔ | ❌ | ❌ | ❔ | ❔ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
<!-- provider-matrix-end -->

Then it’s a simple replace of all the lines between the two markers (matched without using regex, although you could also replace /<!-- provider-matrix-start -->.*<!-- provider-matrix-end -->/ with the updated content)

[tlimoncelli]

With the 'Simple concatenation' solution, the content is also duplicated. What should someone make an adjustment to? In foo.md-top? Or in foo.md? These are steps that need to be described. This also does not seem to me to be a desirable situation.

People would edit the foo.md-top file and then type "go generate" to build foo.md. Yes, that would be a hassle.

[tlimoncelli]

Then it’s a simple replace of all the lines between the two markers (matched without using regex, although you could also replace /<!-- provider-matrix-start -->.*<!-- provider-matrix-end -->/ with the updated content)

Ah, yes! Putting them in a comment makes sense!

We'd use strings.Contains() instead of looking for an exact match.

[systemcrash]

1. Is it an official/community support provider?

2. What are the supported/useable features?

Can/Cannot for providers seems to take care of this, altho the table it produces is a little unwieldy. It's often difficult to see where you are in the matrix - under which heading/provider. Perhaps a tooltip showing that combo is an improvement.

Are there other features that are not revealed in this way that need highlighting?

3. Who is the maintainer?

Does it make sense to put the maintainer in the provider md file? This way there aren't tables in multiple places.

[tlimoncelli]

Yeah, the table is getting unreadable.

There are a couple ways we could re-do it.

1. Split it into separate tables for
   • support (DNS PRovider, Registrar, Official/community)
   • which rtypes are supported (Alias, CAA, etc)
   • features (autodnssec, dual host, create-domains, get-zones, no_purge).
2. Display the current table but remove the left-right scrolling, just have the page be super wide
3. use small columns

I'm open to those plus anything else people suggest.

Does it make sense to put the maintainer in the provider md file? This way there aren't tables in multiple places.

Doesn't matter to me. My primary concern is that it is easy for me to access. For example, I recently CC'ed all maintainers in a PR and I could do that by awking OWNERs. I frequently need to look up "who is the maintainer of x-y-z?" and I do that with OWNERS too.