Local group rate limit is not respected without a default value configured

[vishuvig]

Hi, we're trying to run a set of BullMQ workers with both local group concurrency but are finding that local group rate limits are not enforced unless the worker has a default value set for the group rate limit:

Code snippets and more details to help describe the issue. Our setup is as below:

const generalPurposeQueue = newQueuePro("general-purpose", { connection });

await generalPurposeQueue.setGroupRateLimit("group-id-to-rate-limit", 1, 100000); // once every 100 seconds

const worker = new WorkerPro("general-purpose", processFunc, {
  concurrency: 1, // only job at a time per worker
  group: {
    concurrency: 1, // default 1 job per group across worker pool
  },
  connection,
});

We then have some code to enqueue jobs with the specific group at a rate greater than the rate limit, ie:

...
await generalPurposeQueue.add(
  "general-purpose-job",
  {},
  {
    group: {
      id: "group-id-to-rate-limit"
    }
  }
);
...

Expected Behaviour

Our expectation is that the local group rate limit for group group-id-to-rate-limit would be respected and we should not process more than 1 job every 100 seconds (with the current config).

Actual Behaviour

We do not see any rate limiting and jobs are processed as they are enqueued.

---

We have found a workaround in local development to set a default group rate limit when starting the worker but some concerns with using this as well:

1. When setting a default value for local group rate limit, we are not able to set a default value for the local group concurrency (created separate issue: Local Group Rate Limit and Local Group Concurrency cannot be configured on a BullMQ worker at same time #117) which we also need in our prod env
2. The BullMQ Pro docs don't call out this requirement and limitation as well so I am not sure if this is a bug or designed functionality.

We're on version 7.35.2 of bullmq-pro and version 5.53.1 of bullmq. Let me know if you have any question or anything I can clarify further.

[roggervalf]

hi @vishuvig, what's actually by design to prevent unnecessary checks for groups in a queue that do no need to validate rate limits. We should clarify this behavior in our docs

[vishuvig]

Hi @roggervalf , thanks for confirming that this is the desired behaviour. Would appreciate if you could update the docs and I think it should be fine to close this issue afterwards. Though, on that note, could you please take a look at #117 ? We would like to have both local group rate limit and concurrency config on our worker and are not able to do so currently.