Martin Adámek
Martin Adámek
I wanted to see a repro where the select in strategy is slow, joining to-many relations is destined to be problematic due to the cartesian products, and your repro is...
I wonder what hw you are using, I checked your repro and it behaves twice as fast on my end (M1 pro) :] > 18 empty tables Nope, they are...
few comments from my end: - same issue is likely present in puppeteer, lets fix that too https://github.com/apify/crawlee/blob/master/packages/puppeteer-crawler/src/internals/enqueue-links/click-elements.ts#L557-L561 - some tests would be nice - PR title will end up...
Please address at least the points I made [here](https://github.com/apify/crawlee/pull/2794#issuecomment-2577135434).
There is no proxy on `BasicCrawler` level, this is expected behavior. https://crawlee.dev/js/api/basic-crawler/interface/BasicCrawlerOptions Expected, but I also get confused by this quite often. Maybe we should rework this.
Yeah, I would be for moving the `proxyConfiguration` option to the basic crawler level too, it kinda makes sense if we tell people to use `sendRequest` to have a native...
this might be handy https://github.com/mikro-orm/mikro-orm/blob/master/packages/core/src/utils/Utils.ts#L445-L451
Can you try `pipx run crawlee==0.1.0 create my-crawler`?
I like the JS version, but given many HTTP clients won't support streaming or it will be challenging to implement, I think we could have a default implementation of the...
> Before tackling any new feature I need to figure out how to fix the dataloader for many to many relationships without an inverse side: https://github.com/mikro-orm/mikro-orm/issues/6031 The fix to me...