Feat: return HTML / CSS from scrape and crawl endpoints

[calebpeffer]

For some usecases, it would be valuable to return the raw HTML and CSS without any pre-processing. Discussion on execution below @rafaelsideguide @nickscamara 👇

[nickscamara]

Should it be returned by default? Or should users preference if they want the html? The problem is that some pages are way too long, making the request response "too long" in some cases. We could potentially have some sort of param on the API that users can specify if they want HTML/CSS returned or not.

[nickscamara]

@rafaelsideguide what are your thoughts on this?

[rafaelsideguide]

I think adding an API parameter is the best way to go. This way, Firecrawl won’t get slower for everyone, only for the users who need CSS and HTML in their requests.

[nickscamara]

Hm, I think that's good but because there is not added latency in returning the markdown as well, it might be wise to do so. I was more thinking of an optional to return the html but not exclude the mardown return

[nickscamara]

Maybe change to includeHtml