firecrawl icon indicating copy to clipboard operation
firecrawl copied to clipboard

[Feat] Crawl a website example consumes 16% of the free tier

Open joaoaugustogrobe opened this issue 3 months ago • 1 comments

Problem Description The example for crawl a website shown in the repo description consumes 16%, reading 84 pages.

Proposed Feature Please let's reduce the limit in the example, instead of 100, using a limit of around 5-10 is already enough for demonstrating the feature without consuming that much of the free trial credits for the api.

Implementation Suggestions

# Crawl a website:
crawl_status = app.crawl_url(
  'https://firecrawl.dev', 
  params={
    # - 'limit': 100, 
    'limit': 10, 
    'scrapeOptions': {'formats': ['markdown', 'html']}
  },
  poll_interval=30
)

Additional Context I know this looks super silly, but I was testing the example provided and I in three runs I used most of my free trial, so it's annoying to create another account or self hosting the application before even having a real chance to test the application

joaoaugustogrobe avatar Nov 06 '24 14:11 joaoaugustogrobe