Jindřich Bär

Results 108 comments of Jindřich Bär

This seems to be caused by the following snippet (line 178): https://github.com/apify/crawlee/blob/6f2e6b0ccb404ae66be372e87d762eed67c053bb/packages/core/src/autoscaling/snapshotter.ts#L176-L180 The `availableMemoryRatio` is by default `0.25`, which checks out with our observations. This is probably alright for non-Apify...

This seems to be caused by the latest changes to the `AutoscaledPool` after all ([see the PR](https://github.com/apify/crawlee/pull/2400)) - by calling `addRequests()` in a request handler, we now `notify` the `AutoscaledPool`,...

It seems that the fix didn't help, the issue is persisting in `[email protected]`. ![obrazek](https://github.com/apify/crawlee/assets/61918049/6f7e0659-5130-4ae9-878c-1568450ba41d)

Perhaps we can set the default value of `availableMemoryRatio` in the Apify SDK's `Configuration` ([here](https://github.com/apify/apify-sdk-js/blob/3a3c44054985fe2976d6a0d437e2affae96d3a6c/packages/apify/src/configuration.ts#L98))? We might use the `APIFY_IS_AT_HOME` envvar to switch between the original default and the new...

See https://github.com/apify/crawlee/pull/2425#issuecomment-2061268072 for more `AsyncLocalStorage` experiments.

What Crawlee version are you using? This should be mitigated in the current latest (`3.9.2`).

Yeah, unfortunately, we need the `version_docs` to run conditionally (only on `major` / `minor` releases, not `patch`). The `retry step we use for deploys` sounds - although still a bit...

I mean, if you know where to look, it would be nice if you could 🙏🏽 No worries if you don't find the time, it's quite low-prio and I'll revolve...

Possibly (quite remotely) related to #2317

Seems like a sign of a much larger underlying issue: **New sessions / fingerprints / proxyUrls are generated only on a _browser launch_.** The following snippet doesn't rotate the fingerprints...