mlscraper icon indicating copy to clipboard operation
mlscraper copied to clipboard

Enable increasing complexity in rule-based scraper

Open lorey opened this issue 5 years ago • 1 comments
trafficstars

Currently, the rule-based scraper tries all potential css selectors at once. It would be more performant if we increase the css selector complexity step by step though, so we first try single node rules like div.item and something like .menu > div.item.company if the simpler rules don't work.

lorey avatar Sep 29 '20 09:09 lorey

This is still open, currently rules get generated with more complexity, but increasing complexity stops at selectors with two levels, e.g. .test > .value.target.box.

lorey avatar Jun 21 '22 13:06 lorey