Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobs that do not count to rate limit #2858

Open
sakari opened this issue Oct 23, 2024 · 3 comments
Open

jobs that do not count to rate limit #2858

sakari opened this issue Oct 23, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@sakari
Copy link

sakari commented Oct 23, 2024

Is your feature request related to a problem? Please describe.

I use queue rate limiting to stay within limits of an external API I'm calling. Some jobs however do not require this call, eg the job might be too old and I just want to fail it. These jobs however, as far as I can see, count to the rate limit. So in the worst case a bunch of unimportant NOOP jobs that finish immediately prevent actual useful work from being done. In normal operation this would not be issue -- but in case there are some hickups that stop job processing for a longer duration having to process the stale jobs first would prolong the time to recovery.

Describe the solution you'd like

A way to return from the worker function that makes the job that was processed not count towards rate limits.

@roggervalf
Copy link
Collaborator

hi @sakari, something was investigated some time ago #1653 and also we notice that it would affect rate limit. No sure when we would have time to revive it as it didn't get too much traction

@sakari
Copy link
Author

sakari commented Oct 28, 2024

bummer.

thanks anyway for the reply.

@manast
Copy link
Contributor

manast commented Nov 8, 2024

Unfortunately this is a bit difficult to implement as the rate-limit is updated as soon as job is moved to active status. However we can keep this open as an enhancement as it is in the realms of possibility to implement it some day.

@manast manast added the enhancement New feature or request label Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants