apify-sdk-js
apify-sdk-js copied to clipboard
Respect `ACTOR_MAX_PAID_DATASET_ITEMS` when pushing to dataset
Feature
For PPR actors we need a way to limit the results.
PoC: https://github.com/metalwarrior665/max-paid-items-example/blob/master/src/push-data.ts
Motivation
..
Ideal solution or implementation, and any additional constraints
Respect the ACTOR_MAX_PAID_DATASET_ITEMS env var when calling pushData. Have a custom Dataset implementation for the SDK similar to what we do with KVS, it will override the core method on require time, that way it will work with the crawlee method too (including the one from context to allow adaptive crawling).
Alternative solutions or implementations
No response
Other context
https://apify.slack.com/archives/CD0SF6KD4/p1717532797512629