[Improv] JS SDK checkCrawlStatus returning incomplete response
Describe the Bug
In the API docs, the /crawl/status/{jobId} advertises several parameters, current, total, current_step, and current_url, not present in response via the node sdk. I am seeing only data, partial_data, status, and success in the result for checkCrawlStatus. Would exposing this via JobStatusResponse be easy enough?
To Reproduce Steps to reproduce the issue:
- Initialize an
appobject withnew FirecrawlApp - Run
app.crawlUrl(... - Run await
app.checkCrawlStatus(jobId);withjobIdas the id of the job returned in the above crawl request - Observe fields
current,total,current_step, andcurrent_urlare missing from the response
Expected Behavior
current, total, current_step, and current_url present in the response
Hey @jhoseph88 !
Thanks for pointing out the mismatch in the API docs and the node SDK responses. It looks like we need to update the JobStatusResponse to include the missing fields.
Would you be up for helping us fix this? Your assistance would greatly benefit everyone using the SDK. If you could also update the SDK's end-to-end tests to align with these changes, that would be perfect.
Let me know if you’re interested, or if there’s any other way I can assist.
@rafaelsideguide Pardon my delay! Absolutely--I've submitted https://github.com/mendableai/firecrawl/pull/391. I've had to mark these missing fields as optional because they periodically end up as undefined in my local environment. I'll also point the following tests are failing for me locally:
should throw error for invalid API key on scrapeshould throw error for invalid API key on crawlshould handle idempotency key for crawlshould throw error for invalid API key on search
This is likely due to my configuration.