Provide optional extraction directives

[bejean]

What about provide optional extraction directives ?

In a majority of cases the extraction algorithm woks great. But for some web sites it can fail to extract relevant content. For these web sites it could be possible to "help" snacktory to focus on a specific part of the page content by providing it a Jsoup selector. For instance, we could have something like :

ArticleTextExtractor extractor = new ArticleTextExtractor(); extractor.setTextSelector("div.article_content"); extractor.setTitleSelector("h2", "first"); String dateRegEx = "xxxx"; extractor.setDateSelector("#published", dateRegEx);

JResult res = extractor.extractContent(rawData); text = res.getText(); title = res.getTitle(); date = res.getDate();

[karussell]

Hmmh I don't find this solution that useful as one could simply use jsoup directly for those failing sites. Also I would rather adapt the core to include the failing site. Let me think about it.

[bejean]

Provide a scope to snacktory for the text extraction means to use the snacktory algorithm within this scope. We still need snacktory algorithm.

[karussell]

I see what you mean!