nuxt-seo
nuxt-seo copied to clipboard
Enable crawl via robots but prevent site indexing via noindex
Details
Google mentioned to properly avoid site indexing, the page must not be blocked via robots.txt. It looks like robots plugin isn't working this way.
See important note here: https://developers.google.com/search/docs/crawling-indexing/block-indexing
Currently, i'm only using
site: { indexable: process.env.NUXT_SITE_ENV === "production" }, // false
But it renders both the robots.txt
# START nuxt-simple-robots (indexing disabled)
User-agent: *
Disallow: /
# END nuxt-simple-robots
and the noindex tags
<meta name="robots" content="noindex, nofollow">
Am i doing this correctly, or i missed a configuration?