firecrawl
firecrawl copied to clipboard
[Feat] Crawl a website example consumes 16% of the free tier
Problem Description The example for crawl a website shown in the repo description consumes 16%, reading 84 pages.
Proposed Feature Please let's reduce the limit in the example, instead of 100, using a limit of around 5-10 is already enough for demonstrating the feature without consuming that much of the free trial credits for the api.
Implementation Suggestions
# Crawl a website:
crawl_status = app.crawl_url(
'https://firecrawl.dev',
params={
# - 'limit': 100,
'limit': 10,
'scrapeOptions': {'formats': ['markdown', 'html']}
},
poll_interval=30
)
Additional Context I know this looks super silly, but I was testing the example provided and I in three runs I used most of my free trial, so it's annoying to create another account or self hosting the application before even having a real chance to test the application