getBatch adding same job to several different batches #3097
christhegrand
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
We've been noticing in our logs recently that if getBatch is called concurrently by several different workers, the same job can be added to several different batches.
I would have expected the lock that BullMQ sets in Redis to prevent this from happening. Some of the jobs eventually fail with errors such as
Missing key for job a2a66ac9057404fb2737203d0d058ad08ad7fc360f80a4762205654c2be95338bd69c5b89603768dfc90c2acbf5724606db81d9dcba01046bf524f71691f116d. completed
orLock mismatch for job a2a66ac9057404fb2737203d0d058ad08ad7fc360f80a4762205654c2be95338bd69c5b89603768dfc90c2acbf5724606db81d9dcba01046bf524f71691f116d. Cmd completed from active
, but we are still seeing the same jobs getting processed more than once. This is not caused by stalling, which I know the Bull documentation mentions can lead to double processing.Is this the intended behavior? If so, is there a workaround for this case to prevent several different workers from processing the same jobs?
I am seeing this in BullMQ Pro 7.17.2, which I know is a bit out of date now, but I do not see anything that looks like it might fix this issue in later releases in the CHANGELOG.
Beta Was this translation helpful? Give feedback.
All reactions