Ali Khosravi

@ahkole tests pass now :man_shrugging: I didn't understand in the end what caused the issue. I added this incident to our flaky test collector. For me this PR looks good...

Thanks a lot @sphuber for merging #6517, the file is now placed on this [https://aiida.readthedocs.io/projects/aiida-core/en/latest/robots.txt](https://aiida.readthedocs.io/projects/aiida-core/en/latest/robots.txt) url, which still is not discoverable by search engines. Should be [https://aiida.readthedocs.io/robots.txt](https://aiida.readthedocs.io/robots.txt) . See [here](https://github.com/readthedocs/readthedocs.org/blob/a309d43c136b6466e2f748c4027eea4636ecf349/readthedocs/proxito/views/serve.py#L736),...

Makes sense, patch release is even better! Then I close now, if the issue persist later we can reopen.

Checking after release `v2.6.2`: https://aiida.readthedocs.io/robots.txt is still the old version. New version is located here: https://aiida.readthedocs.io/projects/aiida-core/en/stable/robots.txt which is not discoverable by search engines. We need to understand how to move...

https://support.google.com/webmasters/answer/7489871?hl=en#zippy=%2Cthis-is-my-site As google says, this happens when `robots.txt` is not discoverable 

Historical versions should not be indexed: 

Should have not been indexed if `robots.txt` was discoverable: 


Thanks a lot @eimrek for writing this up. Your suggestion actually makes sense to me. A bigger issue is that `aiida` as in readthedocs is building from this repo: https://github.com/aiidateam/aiida-metapkg...

> @khsrali ok, if you're able to get it to work, that's fine. Nah, actually redirect didn't work :sob: I've put it there, but still is going to the old...