Being able to notify the job/queue to avoid stalling when job is CPU intensive

[maxime-dupuis]

Is your feature request related to a problem? Please describe.

My CPU intensive jobs would do "correct" work, but they stall with errors:

• Missing lock for job JOB_ID
• Error: job stalled more than allowable limit

async checkStaleCreatives(): Promise<any> {
    // Keep busy
    const startTime = Date.now();
    while (Date.now() - startTime < 30000) {
        // Spin
    }
}

Describe the solution you'd like

I'd like a way to tell BullMQ that my job is still working correctly. So that my job is not considered stalled.

await job.notifyNotStalled()
or
await queue.notifyNotStalled()

AND/OR the snippet below (or a better fix) should be in the documentation about stalled jobs so people know how to prevent the stalls

• https://docs.bullmq.io/guide/workers/stalled-jobs
• https://docs.bullmq.io/guide/jobs/stalled
• https://docs.bullmq.io/guide/troubleshooting#missing-locks

Describe alternatives you've considered

Adding this in the CPU intensive loop solves my problem

// Yield to the event loop to avoid stalls
await new Promise((resolve) => setTimeout(resolve));

[roggervalf]

hi @maxime-dupuis sorry for the delay. Looks like we need to improve our docs a little bit. We have these extra options available in Worker class https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#lockDuration and https://api.docs.bullmq.io/interfaces/v5.WorkerOptions.html#lockRenewTime that you can increase. You can make some tries to check which values work for your use cases.

[maxime-dupuis]

lockDuration

Thanks, I think it might be useful sometimes if the expected duration is known.

In our case, the "yielding" trick works best for us because

• We can't predict the duration of the job (not obvious from my code example, sorry!)
• We have natural points where it makes sense to "yield" (every loop)