Robots `noindex` meta tag output in multilingual site

[jpanther]

This issue needs investigation to verify if/when it occurs. Potentially an issue in the theme when multilingual sites are built as there's nothing in the layouts that should be adding this.

I tried with both robots parameter activated and deactivated. And the result didn't change.

I seem to have solved the problem for now: I found out that when I have the two locales (en and it) activated and all pages split in subdirectories (content/en and content/it) hugo builds an index.html file in the root of the site (the public folder) with just a few lines of code, one of which is the robots meta tag set on noindex. That index.html file just stays in the folder and blocks the robots. And there is no way to prevent hugo from building it.

After I put the EN locale site files in the root content directory (content) it all worked and Google was able to index it. Not sure why is that. I also had to deactivate the 2 languages options in the config directory. and I am using translationKey to connect the pages in the 2 languages

Discussed in https://github.com/jpanther/congo/discussions/710

^{Originally posted by giacomosepe November 20, 2023} Hi! I have finished my personal website. And I couldn't be happier of the result. I have installed it using modules and I have set it up with two languages so that now I have EN on mydomain.com/en and IT on mydomain.com/it for every page. It works perfectly. I have deployed with Cloudlfare pages. I have set "enableRobotsTXT = true" and HUGO_ENV=production on Cloudflare variables on deploy.

But after 24hrs Google still throws me a message of error and it can't index the site: " 'noindex' detected in 'robots' meta tag ". I could not find any meta tag in the page code source: giacomosepe.com

I have run out of resources and can't find solutions online. Any ideas of what I am doing wrong? any fixes? I am not a developer but can hustle around a little.

[wolfspyre]

may have some relevance: https://mertbakir.gitlab.io/hugo/sitemap-robots/

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. If you are still experiencing this issue, please review the issue history and add a reply with any requested and/or additional information in order to keep the issue open. This issue will automatically close in 30 days if no further activity occurs.

[jpanther]

Having done some more reading on this and not having anyone else add further discussion I think it's not a big issue. Looking at https://discourse.gohugo.io/t/noindex-complaint-by-google/49038 suggests it's potentially just the root index page that has got the noindex flag in which case this shouldn't be an issue.