Josef Procházka
Josef Procházka
Improved caching was to a certain extent applied to the `ApifyRequestQueueClientSimple` which does not need to handle a multiple consumers scenario https://github.com/apify/apify-sdk-python/pull/573 , since it will be the default client...
https://github.com/apify/apify-sdk-python/pull/573
I also considered this approach initially, but I did not think it was a good one for the following reasons: - The "big class" contains methods and attributes that no...
Either we do some internal refactoring to make it available or we wait for Playwright support. Feature is tagged with 1.52 https://github.com/microsoft/playwright/issues/16239 (upcoming release) We have to wait for such...
This is slightly breaking change, so probably wait until more braking changes accumulate.
New release is coming, so let's add this change in now.
One option to deal with this would be to add some short sleep in _worker_task_orchestrator not call [_worker_task_orchestrator while loop](https://github.com/apify/crawlee-python/blob/master/src/crawlee/_autoscaling/autoscaled_pool.py#L221) so frequently and allow the possibility to add more than...
Hello, this could be connected to the https://github.com/apify/crawlee-python/issues/1224 > I am seeing this log which makes me suspicious crawlee doesn't actually know the memory and cpu it is using: current_concurrency...
> But, either way, if my pod is being killed because it is using more than (or close to) 100% of the memory it is allowed to use, shouldn't crawlee...
> I am not sure it is related, but I am not overriding available_memory_ratio default value in the configuration, so shouldn't crawlee target to using 25% of the available memory?...