Batch size limit in BatchingHttpInterceptor is not respected, causing HTTP 413 error

[mengdd]

Version

4.2

Summary

When using batching as documented:

apolloClientBuilder.httpBatching(
    maxBatchSize = 10
)

We expected the requests batched would be at most of size 10, but in reality it could exceed the size limit and causing error.

Steps to reproduce the behavior

Steps to Reproduce:

• Set maxBatchSize = 10
• Fire 20 concurrent requests quickly
• Observe the 413 error.

Checking the source code here: BatchingHttpInterceptor

We found when the batch is full and wants to send the requests, the lock actually could not be acquired, due to its still be owned by more requests doing adding to the list.

[SanshanGan]

Here are some additional details we observed:

Summary: Under high concurrent request load, a race condition can occur where the actual batch sent contains significantly more requests than the configured maxBatchSize.

1. When the number of pendingRequests reaches maxBatchSize within the intercept method, the sendNow flag is set to true, and executePendingRequests() is called immediately.
2. executePendingRequests() is invoked, and it will eventually run on a single-thread dispatcher.
3. There's a window between the intercept method deciding to send a batch (and releasing its mutex lock, the Lock1) and executePendingRequests() acquiring its mutex lock (the Lock2) to copy and clear the pendingRequests list.
4. During this window, new incoming requests can still invoke intercept(), acquire Lock1, add themselves to the pendingRequests list, and also decide to call executePendingRequests() if the (now larger) list size still exceeds maxBatchSize.
5. Consequently, by the time an instance of executePendingRequests() acquires Lock2 and copies the pendingRequests list, the list may have grown considerably beyond the intended maxBatchSize due to these intervening additions.

[BoD]

Fixed in #6528, thanks!

[github-actions]

Do you have any feedback for the maintainers? Please tell us by taking a one-minute survey. Your responses will help us understand Apollo Kotlin usage and allow us to serve you better.