fess icon indicating copy to clipboard operation
fess copied to clipboard

Add WebFsIndexHelper for web and filesystem crawling

Open marevol opened this issue 1 month ago • 0 comments

Major improvements:

  • Extracted common filter processing logic into reusable methods (processFilters, addFailureExclusionFilters)
  • Extracted crawler setup logic into dedicated methods (setupWebCrawler, setupFileCrawler)
  • Extracted crawler execution logic into executeCrawlers method
  • Reduced code duplication by ~250 lines

Code quality improvements:

  • Fixed incorrect break statements (should be continue)
  • Removed unused code (ComponentUtil.getFessConfig() call)
  • Simplified AtomicBoolean usage in filter processing
  • Improved error messages with config names
  • Fixed issue where session IDs were added even for invalid configs

The doCrawl method is now much more readable and maintainable, going from ~380 lines to ~65 lines while preserving all functionality.

marevol avatar Nov 17 '25 20:11 marevol