Skip to content

Respect ACTOR_MAX_PAID_DATASET_ITEMS when pushing to dataset #304

@B4nan

Description

@B4nan

Feature

For PPR actors we need a way to limit the results.

PoC: https://github.com/metalwarrior665/max-paid-items-example/blob/master/src/push-data.ts

Motivation

..

Ideal solution or implementation, and any additional constraints

Respect the ACTOR_MAX_PAID_DATASET_ITEMS env var when calling pushData. Have a custom Dataset implementation for the SDK similar to what we do with KVS, it will override the core method on require time, that way it will work with the crawlee method too (including the one from context to allow adaptive crawling).

Alternative solutions or implementations

No response

Other context

https://apify.slack.com/archives/CD0SF6KD4/p1717532797512629

Metadata

Metadata

Assignees

Labels

t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions