firecrawl
firecrawl copied to clipboard
[Self-Host] /v1/crawl/ endpoint error when no supabase
Describe the Issue
I have a docker compose setup as described in SELF_HOST.md
. I have not configured supabase or authentication.
When I hit the crawl status endpoint I see the following error:
error [:]: Error occurred in request! (/v1/crawl/6ebe383b-742b-4098-9c41-a1213cdce1df) -- ID 35396e67f7b54070855f1eec0ac31994 -- {"message":"Supabase client is not configured.","name":"Error","stack":"Error: Supabase client is not configured.\n at Proxy.<anonymous> (/app/dist/src/services/supabase.js:41:23)\n at crawlStatusController (/app/dist/src/controllers/v1/crawl-status.js:154:14)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"} {}
To Reproduce Steps to reproduce the issue:
- Configure the environment or settings without SUPABASE_URL (or I am using an empty value)
- Start the stack with docker compose up
- POST to http://localhost:3002/v1/crawl with a valid job
- GET status endpoint with ID from previous step: http://localhost:3002/v1/crawl/6ebe383b-742b-4098-9c41-a1213cdce1df
- Observe the error on the api service container: see description above
Expected Behavior I expect a status 200 and a crawl result.
Environment (please complete the following information):
- OS: docker on WSL
- Firecrawl Version: 5894076
- Node.js Version: node:23-slim
- Docker Version (not applicable): 27.4.0
- Database Type and Version: n/a
Logs
error [:]: Error occurred in request! (/v1/crawl/6ebe383b-742b-4098-9c41-a1213cdce1df) -- ID 35396e67f7b54070855f1eec0ac31994 -- {"message":"Supabase client is not configured.","name":"Error","stack":"Error: Supabase client is not configured.\n at Proxy.<anonymous> (/app/dist/src/services/supabase.js:41:23)\n at crawlStatusController (/app/dist/src/controllers/v1/crawl-status.js:154:14)\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"} {}
Additional Context
I have solved the issue by using the following check, going off what is done in other endpoints (crawlCancelController
for example) this seems the right thing to do when no supabase is configured:
const useDbAuthentication = process.env.USE_DB_AUTHENTICATION === "true";
if (useDbAuthentication && totalCount === 0) {
const x = await supabase_service
.from('firecrawl_jobs')
.select('*', { count: 'exact', head: true })
.eq("crawl_id", req.params.jobId)
.eq("success", true)
totalCount = x.count ?? 0;
}
at https://github.com/mendableai/firecrawl/blob/5894076fda71ef320dcb0e7dcee34ce589059522/apps/api/src/controllers/v1/crawl-status.ts#L247-L255