Rafael Miller
Rafael Miller
Closing this one for now, as we've made several improvements to the concurrency system. Feel free to reopen if you continue to experience any problems.
@Sanix-Darker feel free to assign me for reviewing when it's done!
Hey @Sanix-Darker, I encountered some errors while testing this PR. Here are the logs: ```bash 2024-08-12 18:08:59 node:internal/modules/cjs/loader:1148 2024-08-12 18:08:59 throw err; 2024-08-12 18:08:59 ^ 2024-08-12 18:08:59 2024-08-12 18:08:59 Error:...
@matsubo, I don't understand the need for it. Could we just use 3002 instead of 8181?
@Z-ZHHH can you try with a higher waitFor? like 5000? Usually javascript rendered pages takes more than 3 seconds to load.
That would be awesome!!
@nickscamara not sure if this works yet
Currently, Firecrawl can handle authentications using the `pageOptions.headers` parameter (check the docs [here](https://docs.firecrawl.dev/api-reference/endpoint/scrape)). However, it does not yet support multiple authentications involving different cookies. We are developing SmartCrawl, which will...
@rkroelin I quickly checked the page, and it seems like the links you're interested in are not children of the original page. For example, https://spp.org/western-services-documents?id=296577#:~:text=Markets%2B%20Governance%20Nominations is not a child link...
I'm closing this issue due to inactivity. @rkroelin, if you have any further comments or need to revisit this, feel free to reopen it. Thank you!