ExactlyOnceMiddleware can result in not deleting SQS message (without indication)

[nohagan-gs]

Describe the bug
This is a consequence of quirky SQS behavior so i'm hesitant to call this a JustSaying issue but wanted to open the discussion and determine if this is something that needs to be considered .....

Assertions :

• It is possible for SQS to deliver duplicate messages (when not a FIFO queue with de-duplication)
• SQS's DeleteMessage API can return success but fail to delete the message if the receipt handle is valid but not the latest one for a message (see additional context below)
• Duplicate messages will have different receipt handles

This can result in a message not being deleted from SQS if ....

• The first receive of the message acquires a temporary lock and begins processing
• The duplicate is received and fails to acquire the temporary lock (due to the first still processing)

Steps To reproduce
See attached code .... not really a test but it illustrates the flow and that the DeleteMessage API is only called with the First ReceiptHandle.
SqsDoubleReceiveWhileProcessingTest.zip

Flow :

1. Message Sent
2. [Receiver 1] Message Received - Receipt Handle ABC
3. [Receiver 1] ExactlyOnce : Temporary Lock Acquired
4. [Receiver 1] Main Processing started (Handler called)
5. [Receiver 2] Message Received - ReceiptHandle XYZ
6. [Receiver 2] ExactlyOnce : Temporary Lock Acquire FAILED
7. [Receiver 2] SqsPostProcesser : Skipped Message Delete (with latest handle XYZ) as not successful
8. [Receiver 2] Finished
9. [Receiver 1] Main Processing completed
10. [Receiver 1] ExactlyOnce : Permanent Lock Acquired
11. [Receiver 1] SqsPostProcesser : Message Delete "successful" using ReceiptHandle ABC -> Not actually deleted by SQS as ABC is not the latest Receipt Handle for the Message (XYZ is)
12. [Receiver 1] Completed Successfully
13. [SQS] Visibility Timeout on Message -> Returned to Queue / Moved to DLQ

• If returned to queue then the detected Permanent Lock should cause SqsPostProcesser to remove this
• However, if this occurs on the last attempt before the DLQ'ing kicks in then its immediately moved to the DLQ

Expected behaviour
SQS DeleteMessage should be called with the ReceiptHandle of the duplicate message, but only AFTER the original receive has completed and been confirmed as successful.
Note: as above, unclear if this is something JustSaying should handle .... "expected" on the assumption that it should.

Actual behaviour
There is no attempt to delete the message using the ReceiptHandle of the duplicate (and hence latest) message delivery.

System information:

• OS: MAc OSX
• Library Version 7.0.0.0
• .NET version 8.0.401

Additional context

SQS DeleteMessage behavior described above ...
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_DeleteMessage.html

"Each time you receive a message, meaning when a consumer retrieves a message from the queue, it comes with a unique ReceiptHandle. If you receive the same message more than once, you will get a different ReceiptHandle each time. When you want to delete a message using the DeleteMessage action, you must use the ReceiptHandle from the most recent time you received the message. If you use an old ReceiptHandle, the request will succeed, but the message might not be deleted."