firecrawl icon indicating copy to clipboard operation
firecrawl copied to clipboard

[Feat] Improved includeMain content

Open calebpeffer opened this issue 7 months ago • 1 comments

#p3nnywh1stl3 on the discord had a great suggestion for the tags to exclude to get tidy content from a website:

["script", "style", "nav", "header", "footer", ".advertisement", ".sidebar", ".nav", ".menu", "#comments", "img", "a"]

It would be interesting if we added these tags to the onlyMainContent in V1

calebpeffer avatar Jul 20 '24 02:07 calebpeffer