Delay (pause) consumption

[Arctomachine]

I want to pause queue consumption (i.e. stop performing tasks at all and keep remaining in queue).
Use case: consumer executes task and is asked to retry attempt later (limit exceeded, server is busy, etc) after provided amount of time. So there is no reason to execute other tasks too until specified time has not run out yet.
Then what really needs to be done is to return failed task back into queue and stop further consumption.

But how is it done properly? I searched for pausing and delaying queue, but all I found was dead letter method and delayed message plugin. Which requires posting to queue with predefined execution delay, so no way to delay entries that are already in queue.

Here is visual representation of what I want to do

[Zerpet]

It is not possible to "pause" a subscription per-se. However, you can decide when to acknowledge a message, and therefore "pause" acknowledging messages to RabbitMQ. By not acknowledging messages to RabbitMQ, there will be a point where RabbitMQ will stop dispatching messages to your consumer, see the pre-fetch guide. Then, your consumer could hold onto the failed task (message) without acknowledging, retry after X seconds, and acknowledge the message once the remote server has successfully completed the task. In this approach, you don't need to return the failed task back into queue.

If you really want to "pause" consumption, you will have to unsubscribe your consumer and close the channel. After closing the channel, any message in-flight (i.e. not acknowledged by the consumer) will be automatically re-queued. Alternatively, you could "nack" the message to put it back in the queue and wait in your application; however, this will not pause the delivery of messages from RabbitMQ.

[michaelklishin]

@Arctomachine you can cancel your consumer at any moment and re-register it. In addition to the suggestions by @Zerpet, you can

• Set the prefetch to 1 (this is a very conservative and inefficient setting but it may be optimal for your needs)
• Acknowledge only when certain conditions are met

Note that RabbitMQ will enforce an acknowledgement timeout so such pauses should not last for more than a few minutes.

[Arctomachine]

Thanks. I played with it in meantime, I found that prefetch 1 is good solution for my use case. But info about timeouts is good to know

[Zerpet]

I've been informed by @ansd that your use case can be met using our new native AMQP 1.0 protocol in RabbitMQ 4.0. More details here:

https://www.rabbitmq.com/blog/2024/09/02/amqp-flow-control#echo

You still need to "nack" the message to requeue it, if you want to put the task back in the queue. Note that it will be requeued at the back of the queue, altering the original order.