Skip to content

Add retry limiter support #6318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
friscoMad wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Add retry limiter support #6318

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e2eaf8f
Initial version of the RetryRateLimiter API and implementation
friscoMad Jul 18, 2025
1ad33f4
Added executor to handle exceptions
friscoMad Jul 21, 2025
d224ed2
Fix JavaDoc
friscoMad Jul 21, 2025
0255f18
Revert unintended changes
friscoMad Jul 21, 2025
805b228
Address comments
friscoMad Jul 21, 2025
90ceda0
Fix conditions and expected errors
friscoMad Jul 21, 2025
8ca3ae5
Change constructor API to support floats
friscoMad Jul 22, 2025
e1614d0
Moved RetryLimiterExecutor to change it to package private
friscoMad Jul 24, 2025
ea663f5
Improve docs
friscoMad Jul 24, 2025
1fe7617
Address fixes
friscoMad Jul 31, 2025
f49c3bd
Fix style
friscoMad Jul 31, 2025
a8095c8
Merge branch 'main' into ramiro-aparicio/Add-RetryLimiter
jrhee17 Aug 13, 2025
ed307b2
Merge branch 'main' into ramiro-aparicio/Add-RetryLimiter
jrhee17 Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions core/src/main/java/com/linecorp/armeria/client/retry/AbstractRetryingClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
import com.linecorp.armeria.client.ClientRequestContext;
import com.linecorp.armeria.client.Endpoint;
import com.linecorp.armeria.client.SimpleDecoratingClient;
import com.linecorp.armeria.client.retry.limiter.RetryLimiter;
import com.linecorp.armeria.client.retry.limiter.RetryLimiterExecutor;
import com.linecorp.armeria.common.HttpHeaderNames;
import com.linecorp.armeria.common.HttpRequest;
import com.linecorp.armeria.common.Request;
Expand Down Expand Up @@ -199,8 +201,9 @@ protected final boolean setResponseTimeout(ClientRequestContext ctx) {
* {@code currentAttemptNo} exceeds the {@code maxAttempts} or the {@code nextDelay} is after
* the moment which timeout happens.
*/
protected final long getNextDelay(ClientRequestContext ctx, Backoff backoff) {
return getNextDelay(ctx, backoff, -1);
protected final long getNextDelay(ClientRequestContext ctx, @Nullable RetryLimiter limiter,
Backoff backoff) {
return getNextDelay(ctx, limiter, backoff, -1);
}

/**
Expand All @@ -214,7 +217,8 @@ protected final long getNextDelay(ClientRequestContext ctx, Backoff backoff) {
* the moment which timeout happens.
*/
@SuppressWarnings("MethodMayBeStatic") // Intentionally left non-static for better user experience.
protected final long getNextDelay(ClientRequestContext ctx, Backoff backoff, long millisAfterFromServer) {
protected final long getNextDelay(ClientRequestContext ctx, @Nullable RetryLimiter limiter,
Copy link
Contributor

@schiemon schiemon Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: AbstractRetryingClient is public so this and the change above are breaking changes. Let us note it in the PR description.

Copy link
Contributor Author

@friscoMad friscoMad Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But these are protected methods, do they also count as public api?

Copy link
Contributor

@schiemon schiemon Jul 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this class is also not final people could inherit from it and use those methods. I cannot think of a use-case for it but theoretically it is possible. In the doc comments we are also not advising against it 🤷

Backoff backoff, long millisAfterFromServer) {
requireNonNull(ctx, "ctx");
requireNonNull(backoff, "backoff");
final State state = state(ctx);
Expand All @@ -237,6 +241,9 @@ protected final long getNextDelay(ClientRequestContext ctx, Backoff backoff, lon
return -1;
}

if (!RetryLimiterExecutor.shouldRetry(limiter, ctx, currentAttemptNo)) {
Copy link
Contributor

@schiemon schiemon Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in Armeria it is a convention to put the ctx as the first argument

return -1;
Copy link
Contributor Author

@friscoMad friscoMad Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have metrics about this but afaik it does not seem easy to add metrics here, maybe we could add some debug logging?

}
return nextDelay;
}

Expand Down
35 changes: 27 additions & 8 deletions core/src/main/java/com/linecorp/armeria/client/retry/RetryConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.linecorp.armeria.client.retry.limiter.RetryLimiter;
import com.linecorp.armeria.common.HttpResponse;
import com.linecorp.armeria.common.Response;
import com.linecorp.armeria.common.RpcResponse;
Expand Down Expand Up @@ -87,25 +88,30 @@ static <T extends Response> RetryConfigBuilder<T> builder0(
private final RetryRule fromRetryRuleWithContent;
@Nullable
private RetryRuleWithContent<T> fromRetryRule;
@Nullable
private RetryLimiter retryLimiter;

RetryConfig(RetryRule retryRule, int maxTotalAttempts, long responseTimeoutMillisForEachAttempt) {
this(requireNonNull(retryRule, "retryRule"), null,
maxTotalAttempts, responseTimeoutMillisForEachAttempt, 0);
RetryConfig(RetryRule retryRule, @Nullable RetryLimiter retryLimiter, int maxTotalAttempts,
Copy link
Contributor

@schiemon schiemon Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the change below are also breaking

long responseTimeoutMillisForEachAttempt) {
this(requireNonNull(retryRule, "retryRule"), null, retryLimiter,
maxTotalAttempts, responseTimeoutMillisForEachAttempt, 0);
checkArguments(maxTotalAttempts, responseTimeoutMillisForEachAttempt);
}

RetryConfig(
RetryRuleWithContent<T> retryRuleWithContent,
@Nullable RetryLimiter retryLimiter,
int maxContentLength,
int maxTotalAttempts,
long responseTimeoutMillisForEachAttempt) {
this(null, requireNonNull(retryRuleWithContent, "retryRuleWithContent"),
maxTotalAttempts, responseTimeoutMillisForEachAttempt, maxContentLength);
this(null, requireNonNull(retryRuleWithContent, "retryRuleWithContent"), retryLimiter,
maxTotalAttempts, responseTimeoutMillisForEachAttempt, maxContentLength);
}

private RetryConfig(
@Nullable RetryRule retryRule,
@Nullable RetryRuleWithContent<T> retryRuleWithContent,
@Nullable RetryLimiter retryLimiter,
int maxTotalAttempts,
long responseTimeoutMillisForEachAttempt,
int maxContentLength) {
Expand All @@ -120,6 +126,7 @@ private RetryConfig(
} else {
fromRetryRuleWithContent = RetryRuleUtil.fromRetryRuleWithContent(retryRuleWithContent);
}
this.retryLimiter = retryLimiter;
}

private static void checkArguments(int maxTotalAttempts, long responseTimeoutMillisForEachAttempt) {
Expand All @@ -145,9 +152,12 @@ public RetryConfigBuilder<T> toBuilder() {
assert retryRule != null;
builder = builder0(retryRule);
}
return builder
.maxTotalAttempts(maxTotalAttempts)
.responseTimeoutMillisForEachAttempt(responseTimeoutMillisForEachAttempt);
builder.maxTotalAttempts(maxTotalAttempts)
.responseTimeoutMillisForEachAttempt(responseTimeoutMillisForEachAttempt);
if (retryLimiter != null) {
builder.limiter(retryLimiter);
}
return builder;
}

/**
Expand Down Expand Up @@ -183,6 +193,15 @@ public RetryRuleWithContent<T> retryRuleWithContent() {
return retryRuleWithContent;
}

/**
* Returns the {@link RetryLimiter} which was specified with
* {@link RetryConfigBuilder#limiter(RetryLimiter)}.
*/
@Nullable
public RetryLimiter retryLimiter() {
return retryLimiter;
}

/**
* Returns the {@code maxContentLength}, which is non-zero only if a {@link RetryRuleWithContent} is used.
*/
Expand Down
21 changes: 19 additions & 2 deletions core/src/main/java/com/linecorp/armeria/client/retry/RetryConfigBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import com.google.common.base.MoreObjects;
import com.google.common.base.MoreObjects.ToStringHelper;

import com.linecorp.armeria.client.retry.limiter.RetryLimiter;
import com.linecorp.armeria.common.Flags;
import com.linecorp.armeria.common.Response;
import com.linecorp.armeria.common.annotation.Nullable;
Expand All @@ -37,6 +38,8 @@ public final class RetryConfigBuilder<T extends Response> {
private int maxTotalAttempts = Flags.defaultMaxTotalAttempts();
private long responseTimeoutMillisForEachAttempt = Flags.defaultResponseTimeoutMillis();
private int maxContentLength;
@Nullable
private RetryLimiter retryLimiter;

@Nullable
private final RetryRule retryRule;
Expand All @@ -50,6 +53,7 @@ public final class RetryConfigBuilder<T extends Response> {
this.retryRule = requireNonNull(retryRule, "retryRule");
retryRuleWithContent = null;
maxContentLength = 0;
retryLimiter = null;
}

/**
Expand All @@ -59,6 +63,7 @@ public final class RetryConfigBuilder<T extends Response> {
retryRule = null;
this.retryRuleWithContent = requireNonNull(retryRuleWithContent, "retryRuleWithContent");
maxContentLength = Integer.MAX_VALUE;
retryLimiter = null;
}

/**
Expand Down Expand Up @@ -111,16 +116,27 @@ public RetryConfigBuilder<T> responseTimeoutForEachAttempt(Duration responseTime
return this;
}

/**
* Sets the specified {@link RetryLimiter} to be used for retry budgeting.
*/
public RetryConfigBuilder<T> limiter(RetryLimiter retryLimiter) {
requireNonNull(retryLimiter, "retryLimiter");
this.retryLimiter = retryLimiter;
return this;
}

/**
* Returns a newly-created {@link RetryConfig} from this {@link RetryConfigBuilder}'s values.
*/
public RetryConfig<T> build() {
if (retryRule != null) {
return new RetryConfig<>(retryRule, maxTotalAttempts, responseTimeoutMillisForEachAttempt);
return new RetryConfig<>(retryRule, retryLimiter, maxTotalAttempts,
responseTimeoutMillisForEachAttempt);
}
assert retryRuleWithContent != null;
return new RetryConfig<>(
retryRuleWithContent,
retryLimiter,
maxContentLength,
maxTotalAttempts,
responseTimeoutMillisForEachAttempt);
Expand All @@ -139,6 +155,7 @@ ToStringHelper toStringHelper() {
.add("retryRuleWithContent", retryRuleWithContent)
.add("maxTotalAttempts", maxTotalAttempts)
.add("responseTimeoutMillisForEachAttempt", responseTimeoutMillisForEachAttempt)
.add("maxContentLength", maxContentLength);
.add("maxContentLength", maxContentLength)
.add("retryLimiter", retryLimiter);
}
}
31 changes: 20 additions & 11 deletions core/src/main/java/com/linecorp/armeria/client/retry/RetryingClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
import com.linecorp.armeria.client.ClientRequestContext;
import com.linecorp.armeria.client.HttpClient;
import com.linecorp.armeria.client.ResponseTimeoutException;
import com.linecorp.armeria.client.retry.limiter.RetryLimiter;
import com.linecorp.armeria.client.retry.limiter.RetryLimiterExecutor;
import com.linecorp.armeria.common.AggregatedHttpResponse;
import com.linecorp.armeria.common.AggregationOptions;
import com.linecorp.armeria.common.HttpHeaderNames;
Expand Down Expand Up @@ -330,6 +332,8 @@ private void doExecute0(ClientRequestContext ctx, HttpRequestDuplicator rootReqD

final RetryConfig<HttpResponse> config = mappedRetryConfig(ctx);
if (!ctx.exchangeType().isResponseStreaming() || config.requiresResponseTrailers()) {
RetryLimiterExecutor.onCompletedAttempt(config.retryLimiter(), ctx, ctx.log().partial(),
Copy link
Contributor Author

@friscoMad friscoMad Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this is the correct place to hook; it seems to work, but the logic is convoluted so maybe there is a better place to do it.
In fact I think that this should be in the handle of the aggregate but not 100%.

Copy link
Contributor

@schiemon schiemon Jul 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with that being convoluted; I am about to clean the retrying clients here: #6292 (on halt atm because I have exams)
So I think we can let that PR tackle this

In the end we want to have only one single call to onCompletedAttempt/ have it centralized

Copy link
Contributor

@schiemon schiemon Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@friscoMad At this point where you put the onCompletedAttempt we just initiated the request. It is almost always not finished at this point. As it should be

Invoked when an attempt completes

we may want to put it as a listener to the whenCompleted future on the attempt response

totalAttempts);
response.aggregate().handle((aggregated, cause) -> {
if (cause != null) {
derivedCtx.logBuilder().endRequest(cause);
Expand All @@ -346,6 +350,8 @@ private void doExecute0(ClientRequestContext ctx, HttpRequestDuplicator rootReqD
return null;
});
} else {
RetryLimiterExecutor.onCompletedAttempt(config.retryLimiter(), ctx, ctx.log().partial(),
totalAttempts);
handleStreamingResponse(config, ctx, rootReqDuplicator, originalReq, returnedRes,
future, derivedCtx, response);
}
Expand All @@ -362,10 +368,11 @@ private void handleResponseWithoutContent(RetryConfig<HttpResponse> config, Clie
}
try {
final RetryRule retryRule = retryRule(config);
final RetryLimiter limiter = config.retryLimiter();
final CompletionStage<RetryDecision> f = retryRule.shouldRetry(derivedCtx, responseCause);
f.handle((decision, shouldRetryCause) -> {
warnIfExceptionIsRaised(retryRule, shouldRetryCause);
handleRetryDecision(decision, ctx, derivedCtx, rootReqDuplicator,
handleRetryDecision(decision, limiter, ctx, derivedCtx, rootReqDuplicator,
originalReq, returnedRes, future, response);
return null;
});
Expand Down Expand Up @@ -412,8 +419,9 @@ private void handleStreamingResponse(RetryConfig<HttpResponse> retryConfig, Clie
.handle((decision, cause) -> {
warnIfExceptionIsRaised(ruleWithContent, cause);
truncatingHttpResponse.abort();
handleRetryDecision(decision, ctx, derivedCtx, rootReqDuplicator,
originalReq, returnedRes, future, duplicated);
handleRetryDecision(decision, retryConfig.retryLimiter(), ctx,
derivedCtx, rootReqDuplicator, originalReq,
returnedRes, future, duplicated);
return null;
});
} catch (Throwable cause) {
Expand Down Expand Up @@ -449,9 +457,9 @@ private void handleAggregatedResponse(RetryConfig<HttpResponse> retryConfig, Cli
ruleWithContent.shouldRetry(derivedCtx, aggregatedRes.toHttpResponse(), null)
.handle((decision, cause) -> {
warnIfExceptionIsRaised(ruleWithContent, cause);
handleRetryDecision(
decision, ctx, derivedCtx, rootReqDuplicator, originalReq,
returnedRes, future, aggregatedRes.toHttpResponse());
handleRetryDecision(decision, retryConfig.retryLimiter(), ctx, derivedCtx,
rootReqDuplicator, originalReq, returnedRes, future,
aggregatedRes.toHttpResponse());
return null;
});
} catch (Throwable cause) {
Expand Down Expand Up @@ -521,14 +529,15 @@ private static void handleException(ClientRequestContext ctx,
ctx.logBuilder().endResponse(cause);
}

private void handleRetryDecision(@Nullable RetryDecision decision, ClientRequestContext ctx,
ClientRequestContext derivedCtx, HttpRequestDuplicator rootReqDuplicator,
HttpRequest originalReq, HttpResponse returnedRes,
CompletableFuture<HttpResponse> future, HttpResponse originalRes) {
private void handleRetryDecision(@Nullable RetryDecision decision, @Nullable RetryLimiter limiter,
ClientRequestContext ctx, ClientRequestContext derivedCtx,
HttpRequestDuplicator rootReqDuplicator, HttpRequest originalReq,
HttpResponse returnedRes, CompletableFuture<HttpResponse> future,
HttpResponse originalRes) {
final Backoff backoff = decision != null ? decision.backoff() : null;
if (backoff != null) {
final long millisAfter = useRetryAfter ? getRetryAfterMillis(derivedCtx) : -1;
final long nextDelay = getNextDelay(ctx, backoff, millisAfter);
final long nextDelay = getNextDelay(ctx, limiter, backoff, millisAfter);
if (nextDelay >= 0) {
abortResponse(originalRes, derivedCtx);
scheduleNextRetry(
Expand Down
5 changes: 4 additions & 1 deletion core/src/main/java/com/linecorp/armeria/client/retry/RetryingRpcClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import com.linecorp.armeria.client.ResponseTimeoutException;
import com.linecorp.armeria.client.RpcClient;
import com.linecorp.armeria.client.endpoint.EndpointGroup;
import com.linecorp.armeria.client.retry.limiter.RetryLimiterExecutor;
import com.linecorp.armeria.common.HttpRequest;
import com.linecorp.armeria.common.Request;
import com.linecorp.armeria.common.RpcRequest;
Expand Down Expand Up @@ -195,10 +196,12 @@ private void doExecute0(ClientRequestContext ctx, RpcRequest req,
res.handle((unused1, cause) -> {
try {
assert retryRule != null;
RetryLimiterExecutor.onCompletedAttempt(retryConfig.retryLimiter(), ctx, ctx.log().partial(),
totalAttempts);
retryRule.shouldRetry(derivedCtx, res, cause).handle((decision, unused3) -> {
final Backoff backoff = decision != null ? decision.backoff() : null;
if (backoff != null) {
final long nextDelay = getNextDelay(derivedCtx, backoff);
final long nextDelay = getNextDelay(derivedCtx, retryConfig.retryLimiter(), backoff);
if (nextDelay < 0) {
onRetryComplete(ctx, derivedCtx, res, future);
return null;
Expand Down
Loading
Loading