Create robots.txt

[ibrand]

De-index entire archval site from google. robots.txt is the softest way to block the indexing and still might not work in some cases -- to fully get rid of the indexing you'd want to use: <meta name="robots" content="noindex"> -- Explained here: https://stackoverflow.com/a/47083230

But I think this would likely work well enough. It has for my personal site so far. It'll take a little troubleshooting to verify that this worked but I can submit another PR if adjustments need to be made.

One caution: if this does work, google wont recrawl in the future (though i think you can submit a re-crawl request) so best to make sure you actually want the full site de-indexed from the search results.

[netlify[bot]]

❌ Deploy Preview for sfpc failed.

Name	Link
🔨 Latest commit	ebb37613378979a24ffdadeaec6404f629a3a9c7
🔍 Latest deploy log	https://app.netlify.com/sites/sfpc/deploys/64fb63ba8aeeda00081e4df2