quick-crawler
quick-crawler copied to clipboard
java crawler framework
Bumps [jsoup](https://github.com/jhy/jsoup) from 1.9.2 to 1.15.3. Release notes Sourced from jsoup's releases. jsoup 1.15.3 jsoup 1.15.3 is out now, and includes a security fix for potential XSS attacks, along with...
Bumps commons-io from 2.5 to 2.7. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
定时爬取
db存爬取任务的记录在任务量较大时,有风险,考虑采用berkely进行持久化
爬取过程中,如果某一个网址出现异常,则退出方式有问题;且没有重试机智