quick-crawler icon indicating copy to clipboard operation
quick-crawler copied to clipboard

java crawler framework

Results 5 quick-crawler issues
Sort by recently updated
recently updated
newest added

Bumps [jsoup](https://github.com/jhy/jsoup) from 1.9.2 to 1.15.3. Release notes Sourced from jsoup's releases. jsoup 1.15.3 jsoup 1.15.3 is out now, and includes a security fix for potential XSS attacks, along with...

dependencies

Bumps commons-io from 2.5 to 2.7. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=commons-io:commons-io&package-manager=maven&previous-version=2.5&new-version=2.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...

dependencies

db存爬取任务的记录在任务量较大时,有风险,考虑采用berkely进行持久化

爬取过程中,如果某一个网址出现异常,则退出方式有问题;且没有重试机智