a better robots.txt

[craigbox]

Once #16559 is finished, we should know on any given Netlify deploy, if we are "production" or not, and what our baseURL is.

#16558 adds a static robots.txt file. We should be able to update it to something more like

User-agent: *
{{- if eq hugo.Environment "production" }}
Sitemap: {{ "sitemap.xml" | absURL }}
{{- else }}
Disallow: /
{{- end }}

at this point.