apify-sdk-js icon indicating copy to clipboard operation
apify-sdk-js copied to clipboard

Respect `ACTOR_MAX_PAID_DATASET_ITEMS` when pushing to dataset

Open B4nan opened this issue 1 year ago • 0 comments

Feature

For PPR actors we need a way to limit the results.

PoC: https://github.com/metalwarrior665/max-paid-items-example/blob/master/src/push-data.ts

Motivation

..

Ideal solution or implementation, and any additional constraints

Respect the ACTOR_MAX_PAID_DATASET_ITEMS env var when calling pushData. Have a custom Dataset implementation for the SDK similar to what we do with KVS, it will override the core method on require time, that way it will work with the crawlee method too (including the one from context to allow adaptive crawling).

Alternative solutions or implementations

No response

Other context

https://apify.slack.com/archives/CD0SF6KD4/p1717532797512629

B4nan avatar Jun 05 '24 08:06 B4nan