-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
req/should-haveshould be finished before deadlineshould be finished before deadline
Description
A task can fail in several places, and because data can't be submitted atomically across the entire system we need to have a mechanism for detecting data associated with failed processing attempts.
I propose that we generate a random identifier every time a worker receives a task, and once the worker has completed processing of the task it submits the random identifier to a list of successful processing attempts. I suggest calling this random identifier the processing identifier. Any data with a processing ID that isn't in the list of successful processing attempt can be discarded and considered failed.
Metadata
Metadata
Assignees
Labels
req/should-haveshould be finished before deadlineshould be finished before deadline