preact-www icon indicating copy to clipboard operation
preact-www copied to clipboard

chore: Add `noindex` meta tag to v8 doc pages

Open rschristian opened this issue 1 year ago • 0 comments

Closes #910

Re: v8 docs being indexed (often higher than v10).

The meta tag is the correct course, rather than say Disallow in the robots.txt, as Disallow only blocks the crawler from discovering the page while navigating our site. While this often does have an effect, external links to the v8 docs could keep them ranking highly, which would be an issue. What we actually want to do is signal that these pages shouldn't be indexed, not hide it from discovery, hence, the meta tag.

https://developers.google.com/search/docs/crawling-indexing/robots/robots-faq?visit_id=637994089979808237-3634207106&rd=1#h17

~~What I am unsure of is if we need to do client-side addition/removal of this tag upon navigation. I haven't quite been able to get a definitive answer on that, does anyone know?~~ Edit: Upon doing some more reading I don't think that will be an issue.

rschristian avatar Sep 22 '22 02:09 rschristian