fess
fess copied to clipboard
Add WebFsIndexHelper for web and filesystem crawling
Major improvements:
- Extracted common filter processing logic into reusable methods (processFilters, addFailureExclusionFilters)
- Extracted crawler setup logic into dedicated methods (setupWebCrawler, setupFileCrawler)
- Extracted crawler execution logic into executeCrawlers method
- Reduced code duplication by ~250 lines
Code quality improvements:
- Fixed incorrect break statements (should be continue)
- Removed unused code (ComponentUtil.getFessConfig() call)
- Simplified AtomicBoolean usage in filter processing
- Improved error messages with config names
- Fixed issue where session IDs were added even for invalid configs
The doCrawl method is now much more readable and maintainable, going from ~380 lines to ~65 lines while preserving all functionality.