doogle icon indicating copy to clipboard operation
doogle copied to clipboard

How to crawl only the suggested site and not external sites

Open safesploit opened this issue 1 year ago • 0 comments

Discussed in https://github.com/safesploitOrg/doogle/discussions/24

Originally posted by benzvi8888 March 10, 2024 Seems when I checked the database the crawler go into social link and start index their content - it crawl Facebook, Twitter, LinkedIn and so on - it filled my database with it. is there any way/option/code to limit the crawling to the site itself only?

safesploit avatar Mar 13 '24 21:03 safesploit