Rafael Miller

Suggested on #3

solves #48

"Consider adding idempotency feature for our backend POST apis, and allow client to pass an idempotency key to avoid submitting duplicate jobs" Suggested by @by12380 on Discord.

[Doc] Add default value of crawlerOptions.limit to /crawl documentation

1

"Show the default value of crawlerOptions.limit in https://docs.firecrawl.dev/api-reference/endpoint/crawl. Judging from the code itself it looks like it's set as 10000." Suggested by @by12380 on Discord.

"Provide an API to cancel jobs, especially for expensive ones. Setting the default limit at 10,000 could potentially break someone’s bank." Suggested by @by12380 on Discord

User is able to get partial results on /crawl/status route

solves #15

Closes #117

suggested on Discord