doogle
doogle copied to clipboard
How to crawl only the suggested site and not external sites
Discussed in https://github.com/safesploitOrg/doogle/discussions/24
Originally posted by benzvi8888 March 10, 2024 Seems when I checked the database the crawler go into social link and start index their content - it crawl Facebook, Twitter, LinkedIn and so on - it filled my database with it. is there any way/option/code to limit the crawling to the site itself only?