boilerpipe
boilerpipe copied to clipboard
Published
20 hours ago •
k-bx
k-bx
Ignore FORM tags in HTMLHighlighter
When using HTMLHighlighter some times boilerpipe keeps some artifacts related
coming from FORM and LABEL tags.
This can be easily prevented by addding a new ignorable element to TAG_ACTIONS
map in HTMLHighlighter.java:
TAG_ACTIONS.put("FORM", TA_IGNORABLE_ELEMENT);
Original issue reported on code.google.com by [email protected] on 24 Mar 2012 at 6:40
Issue 45 has been merged into this issue.
Original comment by ckkohl79 on 25 Mar 2012 at 2:12
Adding FORM as an ignorable element at the highlighter (but not at the
Extractor itself) has two disadvantages:
1. The highlighted HTML will not be consistent with the TextDocument's content
information (i.e., it should rather go into DefaultTagActionMap)
2. FORM may span over many otherwise relevant text blocks. Adding FORM as
TA_IGNORABLE_ELEMENT significantly reduces extraction accuracy (with L3S-GN1,
avg. token-level F1 goes down by about 5% to 7%).
Could you please give examples (URLs) where boilerpipe fails currently?
Please try the very latest version from trunk, or -- preferably -- the version
at http://boilerpipe-web.appspot.com/
Original comment by ckkohl79 on 25 Mar 2012 at 2:19
- Added labels: Type-Enhancement
- Removed labels: Type-Defect
Adding FORM as an ignorable element at the highlighter (but not at the
Extractor itself) has two disadvantages:
1. The highlighted HTML will not be consistent with the TextDocument's content
information (i.e., it should rather go into DefaultTagActionMap)
2. FORM may span over many otherwise relevant text blocks. Adding FORM as
TA_IGNORABLE_ELEMENT significantly reduces extraction accuracy (with L3S-GN1,
avg. token-level F1 goes down by about 5% to 7%).
Could you please give examples (URLs) where boilerpipe fails currently?
Please try the very latest version from trunk, or -- preferably -- the version
at http://boilerpipe-web.appspot.com/
Original comment by ckkohl79 on 25 Mar 2012 at 2:19