Rafael Miller
Rafael Miller
I'm closing this one.
Closes #163 - [x] /scrape - [x] /crawl
We still need to figure out an autoscaling strategy for /crawl. A solution is to: - Create a cron service that checks every minute the active jobs queue - If...
I tested the `/crawl/status/:jobId` route using the `mendable.ai` URL. While the error appeared, I was still able to retrieve the page content using fetch. Additionally, to verify the docker network...
@JakobStadlhuber Yeah.. I'm still trying to figure out why this is happening.. The proxy is a great idea! Feel free to make the PR! Just make sure that the proxy...
Hey @JakobStadlhuber I think I've found the problem. The env variable for playwright is misconfigured. It should have `/html` at the end. So it should be: `PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/html` I tested it,...
@JakobStadlhuber your playwright service is getting timeout errors. Can you test it on simple pages? For example,`firecrawl.dev` should take less than 15 seconds to scrape.
Hey @stevegardnersa Thanks for reporting this issue. It looks like you're encountering a segmentation fault during the pnpm install step in your Docker build. Can you try stopping, removing, and...
@stevegardnersa the error indicates a typescript conflict due to multiple versions of the @types/node package being installed. I did a improvement on the `main` branch that solves this conflict. Can...
@stevegardnersa awesome! Closing this issue.