Scheduling Crawl jobs? Like cron jobs

[younissk]

Is there some capability to schedule Crawl jobs on the docker service, like cron jobs? E.g. Crawl this website every x minutes or once a day at 5pm? Or does this require a wrapper?

[aniruddhaadak80]

From my point of view, scheduling feels like an orchestration concern rather than a crawling concern unless the product is intentionally becoming a persistent job runner. The crawler itself mainly needs idempotent invocation, clear job status, and a clean API boundary that an external scheduler can drive reliably.

If those primitives are strong, cron-like behavior can stay outside the core without users losing anything important.