Vishant G

Results 2 comments of Vishant G

Refactor the code to make sure os path related logic is OS agnostic

Hi! I’m new to open source and would like to work on this issue as my first contribution. Could you please assign it to me or guide me on how...

AI Crawlers

Hello @kingthorin , I will like to contribute to this issue. I can draft a short section explaining how some AI crawlers ignore robots.txt dissalowed files&folders and why it’s a...