nuxt-seo icon indicating copy to clipboard operation
nuxt-seo copied to clipboard

Enable crawl via robots but prevent site indexing via noindex

Open ralph-burstsms opened this issue 11 months ago • 1 comments

Details

Google mentioned to properly avoid site indexing, the page must not be blocked via robots.txt. It looks like robots plugin isn't working this way.

See important note here: https://developers.google.com/search/docs/crawling-indexing/block-indexing

Currently, i'm only using

  site: { indexable: process.env.NUXT_SITE_ENV === "production" }, // false

But it renders both the robots.txt

# START nuxt-simple-robots (indexing disabled)
User-agent: *
Disallow: /

# END nuxt-simple-robots

and the noindex tags

<meta name="robots" content="noindex, nofollow">

Am i doing this correctly, or i missed a configuration?

ralph-burstsms avatar Apr 01 '24 01:04 ralph-burstsms