feat: add automatic graceful exit handlers for abort and migrating events#561
feat: add automatic graceful exit handlers for abort and migrating events#561
abort and migrating events#561Conversation
…ents Register event handlers in Actor.init() that automatically call Actor.exit() when the platform signals an abort or migration. This ensures graceful shutdown without requiring developers to manually handle these events. Key changes: - Add graceful exit handler for 'aborting' and 'migrating' events - Handler waits 1 second to allow other event handlers (like Crawlee's state persistence) to be triggered first - Add isExiting flag to prevent double-exit when Actor.exit() is already in progress - Add test for the new graceful exit behavior Thread: https://apify.slack.com/archives/C07ED56TA1K/p1771414614569569?thread_ts=1771347955.370739&cid=C07ED56TA1K https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
|
isn't this a breaking change? |
|
How exactly? If one of those events fire, the actor will shut down either way in ~30s. |
Exactly like this, 30s is a lot and I can imagine a lot of Actors relying on this unknowingly. Imagine doing |
|
First of all, I don't think the 30s were ever guaranteed. And if they relied on this, it feels like a bug in their code. IMO it's fine, worst case we can revert it, but it's a much better solution than polluting all the templates with the explicit handler (see the slack conversation for details). I'll add an opt-out of this, too. |
…able - Use setTimeout instead of async sleep in graceful exit handler to avoid deadlock with waitForAllListenersToComplete() in exit() - Remove unnecessary 1s delay (exit() already waits for all handlers) - Add gracefulShutdown option to disable auto-exit on aborting/migrating - Add gracefulShutdownDelayMillis option for configurable delay - Reset isExiting flag after exit() completes to fix singleton reuse - Add tests for migrating event and gracefulShutdown: false option https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
Change gracefulShutdown option from default true (opt-out) to default false (opt-in) to avoid potential breaking changes for existing users. https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
- Add graceful shutdown section to actor_lifecycle.mdx with examples - Update Actor.init() JSDoc to document gracefulShutdown option - Update PlatformEventManager JSDoc to mention gracefulShutdown for aborting/migrating events https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
src/actor.ts
Outdated
| export interface InitOptions { | ||
| storage?: StorageClient; | ||
| /** | ||
| * Whether to automatically call `Actor.exit()` when the platform sends an `aborting` or `migrating` event. |
There was a problem hiding this comment.
This is not correct, migrating needs to trigger Actor.reboot, not Actor.exit. I think I tested it some time ago and it just shuts down forever.
There was a problem hiding this comment.
How come? Actor.exit will just call process.exit, there are no API calls, if the actor is migrating, that's handled by the platform itself, no?
As opposed to Actor.reboot which is an API call to the platform to actually restart the Actor (so something that should be already in progress if you see migrating event).
There was a problem hiding this comment.
I just validated in a test that if you call Actor.exit after receiving migrating, the Actor will simply finish with SUCCEEDED while it should keep running through the migration and after it. Funnily, if you throw, the migration will happen immediately and successfully continue the run.
But the correct recommended behavior is to call Actor.reboot on migrating to speed it up.
Being API call or not doesn't really matter here
|
Regarding the breaking change, I'm leaning slightly to yolo it out before v4. I think the risk of messing up someone is quite low, and we will not need another configuration in the templates.
|
Calling exit() on migration would terminate the run, but migrations should continue on a new worker. reboot() speeds up the migration and lets the run resume. exit() is still used for aborting events. Updated implementation, tests, JSDoc, and docs accordingly. https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
metalwarrior665
left a comment
There was a problem hiding this comment.
Otherwise it looks good
|
|
||
| If you're using Crawlee crawlers (like `CheerioCrawler`, `PlaywrightCrawler`, etc.), graceful shutdown is handled automatically. The crawler listens for these events and stops accepting new requests while finishing the ones in progress. | ||
|
|
||
| ### Without Crawlee crawlers |
There was a problem hiding this comment.
I think the with vs without Crawlee is mixing two unrelated things:
- Crawler stops fetching new requests from the queue. This is mainly to save resources like proxies for requests that would not finish anyway. It might finish the ones in progress till the 30 sec migration.
- SDK (with or without Crawlee) with
gracefulShutdown: truewill exit immediately (no waiting for requests in progress) after persist. This is mainly to make abort faster and sync state better
src/actor.ts
Outdated
| * **Graceful shutdown:** When running on the Apify platform, the Actor may receive `aborting` or `migrating` | ||
| * events. By setting `options.gracefulShutdown` to `true`, the SDK will automatically call `Actor.exit()` | ||
| * on `aborting` events and `Actor.reboot()` on `migrating` events (to speed up the migration and continue the | ||
| * run on a new worker). This is useful for Actors that don't use Crawlee crawlers (which handle this internally) |
There was a problem hiding this comment.
Again, not related to Crawlee?
This is a pure SDK feature - the framing about "with/without Crawlee crawlers" was incorrect and confusing. https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
metalwarrior665
left a comment
There was a problem hiding this comment.
Not sure if you were waiting for my review but it is good by me. As I said above, I would set this to true by default.
Co-authored-by: Lukáš Křivka <lukaskrivka@gmail.com>
|
I was out last week. Let's move this forward. I would also just enable this. If we see real issues reported, we can reconsider and go with enabling this in the templates instead. |
45ab49b to
59dd3e7
Compare
59dd3e7 to
45ab49b
Compare
Changed gracefulShutdown from opt-in to opt-out: - Default is now true (handlers registered automatically) - Set gracefulShutdown: false to disable Updated docs and tests accordingly. https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV
Register event handlers in
Actor.init()that automatically callActor.exit()when the platform signals an abort or migration. This ensures a graceful shutdown without requiring developers to manually handle these events.Key changes:
abortingandmigratingeventsisExitingflag to prevent double-exit whenActor.exit()is already in progressThread: https://apify.slack.com/archives/C07ED56TA1K/p1771414614569569?thread_ts=1771347955.370739&cid=C07ED56TA1K
https://claude.ai/code/session_017U4px7vy75ioiTaC5teCaV