Jindřich Bär

Platform distribution in generative networks

15

Recently, I have come across complaints (ty @B4nan, @AndreyBykov ) about suspicious screen dimensions in the fingerprint-injected browsers. Those complaints were mainly about generating vertical screen dimensions for desktop devices....

HttpCrawler - determining character encoding

1

### Which package is this bug report for? If unsure which one to select, leave blank @crawlee/http (HttpCrawler) ### Issue description The HTML living standard [defines steps](https://html.spec.whatwg.org/#determining-the-character-encoding) for determining the...

### Which package is the feature request for? If unsure which one to select, leave blank @crawlee/core ### Feature As discussed in [this WCC PR](https://github.com/apify/store-website-content-crawler/pull/212), the `enqueueLinks` method (and the...

Mirrors the "Proxy Tiers" feature from Crawlee (see more at https://github.com/apify/crawlee/pull/2348). Adds a shorthand option for generating the "proxy tiers" from user-friendly options (see tests). There might be some issues...

The default value of `availableMemoryRatio` is too low

9

In Apify Actor with **4 GB** of available memory, the `AutoscaledPool` refuses to scale up, as it only sees ~**1 GB** of free memory.  This slows down scrapers and...

Race conditions in CI/CD

4

We currently use workflow dispatch from the `release` job to update the Docker build settings ([here](https://github.com/apify/crawlee/blob/28d83f383af9a154a5c57b84f341df5395cff4dd/.github/workflows/release.yml#L167)). The issue is that the workflow dispatch doesn't wait for the workflow to finish,...

Mysterious timeout hard-kills `CheerioCrawler` script

6

As found in one of the community Actors:  This doesn't seem to be related to the `request timed out after 30 seconds` above - instead, it seems as if...

Better CLI API

7

The current state of the Apify CLI API is dubious. As a user, I’m always doubting the difference between `apify run` and `apify call`, whether `apify create` does something locally...

feat: add `FileDownload` "crawler"

3

Adds a new package `@crawlee/file-download`, which overrides the `HttpCrawler`'s MIME type limitations and allows the users to download arbitrary files. Aside from the regular `requestHandler`, this crawler introduces `streamHandler`, which...