Open-Context-Data icon indicating copy to clipboard operation
Open-Context-Data copied to clipboard

Published 20 hours ago • verified publisher icon ekansa

Why is the project excluded in Github's robots.txt?

Open matiasgarciaisaia opened this issue 4 years ago • 4 comments

Hi there!

Sorry if I add too much noise.

I came across Github's robots.txt file and this project is specifically Denied for bots to crawl.

Do you have any idea why? Have you asked for that, or was it a Github's decision?

Thanks for letting me scratch my curiosity :)

matiasgarciaisaia avatar Apr 13 '20 12:04 matiasgarciaisaia

came here to ask the same question

15696 avatar Nov 02 '20 17:11 15696

I'm here for the same reason 😕

donno2048 avatar Jan 20 '21 21:01 donno2048

also bewildered by this lol

ghost avatar Oct 06 '21 20:10 ghost

https://twitter.com/ekansa/status/1137052076062650368

boehs avatar Jan 31 '23 01:01 boehs