-
Notifications
You must be signed in to change notification settings - Fork 580
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
4.x: Concurrency limits module, and support in Helidon WebServer (#9295)
* Concurrency limits module, and support in Helidon WebServer * Align configuration key for server feature and server. * Refactored to use tokens. * Added tests for configuration based limits. * Added copy to a limit, so we can get another instance with the same configuration. * Added support to bypass queuing and return immediately. Signed-off-by: Tomas Langer <[email protected]> Co-authored-by: André Rouél <[email protected]> Signed-off-by: Tomas Langer <[email protected]>
- Loading branch information
1 parent
bf787bf
commit f39decf
Showing
57 changed files
with
3,079 additions
and
98 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| Concurrency Limits | ||
| ----- | ||
|
|
||
| This module provides concurrency limits, so we can limit the number of concurrent, in-progress operations (for example in WebServer). | ||
|
|
||
| The implemented concurrency limits are: | ||
|
|
||
| | Key | Weight | Description | | ||
| |---------|--------|--------------------------------------------------------------| | ||
| | `fixed` | `90` | Semaphore based concurrency limit, supports queueing | | ||
| | `aimd` | `80` | AIMD based limit (additive-increase/multiplicative-decrease) | | ||
|
|
||
| Current usage: `helidon-webserver` | ||
|
|
||
| The weight is not significant (unless you want to override an implementation using your own Limit with a higher weight), as the usages in Helidon use a single (optional) implementation that must be correctly typed in | ||
| configuration. | ||
|
|
||
| # Fixed concurrency limit | ||
|
|
||
| The fixed concurrency limit is based on a semaphore behavior. | ||
| You can define the number of available permits, then each time a token is requested, a permit (if available) is returned. | ||
| When the token is finished (through one of its lifecycle operations), the permit is returned. | ||
|
|
||
| When the limit is set to 0, an unlimited implementation is used. | ||
|
|
||
| The fixed limit also provides support for defining a queue. If set to a value above `0`, queuing is enabled. In such a case we enqueue a certain number of requests (with a configurable timeout). | ||
|
|
||
| Defaults are: | ||
| - `permits: 0` - unlimited permits (no limit) | ||
| - `queue-length: 0` - no queuing | ||
| - `queue-timeout: PT1S` - 1 second timout in queue, if queuing is enabled | ||
|
|
||
| # AIMD concurrency limit | ||
|
|
||
| The additive-increase/multiplicative-decrease (AIMD) algorithm is a feedback control algorithm best known for its use in TCP congestion control. AIMD combines linear growth of the congestion window when there is no congestion with an exponential reduction when congestion is detected. | ||
|
|
||
| This implementation provides variable concurrency limit with fixed minimal/maximal number of permits. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <!-- | ||
| Copyright (c) 2024 Oracle and/or its affiliates. | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
| <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <parent> | ||
| <groupId>io.helidon.common.concurrency</groupId> | ||
| <artifactId>helidon-common-concurrency-project</artifactId> | ||
| <version>4.2.0-SNAPSHOT</version> | ||
| <relativePath>../pom.xml</relativePath> | ||
| </parent> | ||
|
|
||
| <artifactId>helidon-common-concurrency-limits</artifactId> | ||
| <name>Helidon Common Concurrency Limits</name> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>io.helidon.common</groupId> | ||
| <artifactId>helidon-common</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.builder</groupId> | ||
| <artifactId>helidon-builder-api</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.common</groupId> | ||
| <artifactId>helidon-common-config</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.service</groupId> | ||
| <artifactId>helidon-service-registry</artifactId> | ||
| <optional>true</optional> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.config</groupId> | ||
| <artifactId>helidon-config</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.config</groupId> | ||
| <artifactId>helidon-config-yaml</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.hamcrest</groupId> | ||
| <artifactId>hamcrest-all</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.junit.jupiter</groupId> | ||
| <artifactId>junit-jupiter-api</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-compiler-plugin</artifactId> | ||
| <configuration> | ||
| <annotationProcessorPaths> | ||
| <path> | ||
| <groupId>io.helidon.codegen</groupId> | ||
| <artifactId>helidon-codegen-apt</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </path> | ||
| <path> | ||
| <groupId>io.helidon.builder</groupId> | ||
| <artifactId>helidon-builder-codegen</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </path> | ||
| <path> | ||
| <groupId>io.helidon.config.metadata</groupId> | ||
| <artifactId>helidon-config-metadata-codegen</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </path> | ||
| <path> | ||
| <groupId>io.helidon.codegen</groupId> | ||
| <artifactId>helidon-codegen-helidon-copyright</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </path> | ||
| </annotationProcessorPaths> | ||
| </configuration> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>io.helidon.codegen</groupId> | ||
| <artifactId>helidon-codegen-apt</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.builder</groupId> | ||
| <artifactId>helidon-builder-codegen</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.config.metadata</groupId> | ||
| <artifactId>helidon-config-metadata-codegen</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.helidon.codegen</groupId> | ||
| <artifactId>helidon-codegen-helidon-copyright</artifactId> | ||
| <version>${helidon.version}</version> | ||
| </dependency> | ||
| </dependencies> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> |
140 changes: 140 additions & 0 deletions
140
common/concurrency/limits/src/main/java/io/helidon/common/concurrency/limits/AimdLimit.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| /* | ||
| * Copyright (c) 2024 Oracle and/or its affiliates. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package io.helidon.common.concurrency.limits; | ||
|
|
||
| import java.util.Optional; | ||
| import java.util.concurrent.Callable; | ||
| import java.util.concurrent.Semaphore; | ||
| import java.util.function.Consumer; | ||
|
|
||
| import io.helidon.builder.api.RuntimeType; | ||
| import io.helidon.common.config.Config; | ||
|
|
||
| /** | ||
| * AIMD based limiter. | ||
| * <p> | ||
| * The additive-increase/multiplicative-decrease (AIMD) algorithm is a feedback control algorithm best known for its use in TCP | ||
| * congestion control. AIMD combines linear growth of the congestion window when there is no congestion with an exponential | ||
| * reduction when congestion is detected. | ||
| */ | ||
| @SuppressWarnings("removal") | ||
| @RuntimeType.PrototypedBy(AimdLimitConfig.class) | ||
| public class AimdLimit implements Limit, SemaphoreLimit, RuntimeType.Api<AimdLimitConfig> { | ||
| static final String TYPE = "aimd"; | ||
|
|
||
| private final AimdLimitConfig config; | ||
| private final AimdLimitImpl aimdLimitImpl; | ||
|
|
||
| private AimdLimit(AimdLimitConfig config) { | ||
| this.config = config; | ||
| this.aimdLimitImpl = new AimdLimitImpl(config); | ||
| } | ||
|
|
||
| /** | ||
| * Create a new fluent API builder to construct {@link io.helidon.common.concurrency.limits.AimdLimit} | ||
| * instance. | ||
| * | ||
| * @return fluent API builder | ||
| */ | ||
| public static AimdLimitConfig.Builder builder() { | ||
| return AimdLimitConfig.builder(); | ||
| } | ||
|
|
||
| /** | ||
| * Create a new instance with all defaults. | ||
| * | ||
| * @return a new limit instance | ||
| */ | ||
| public static AimdLimit create() { | ||
| return builder().build(); | ||
| } | ||
|
|
||
| /** | ||
| * Create a new instance from configuration. | ||
| * | ||
| * @param config configuration of the AIMD limit | ||
| * @return a new limit instance configured from {@code config} | ||
| */ | ||
| public static AimdLimit create(Config config) { | ||
| return builder() | ||
| .config(config) | ||
| .build(); | ||
| } | ||
|
|
||
| /** | ||
| * Create a new instance from configuration. | ||
| * | ||
| * @param config configuration of the AIMD limit | ||
| * @return a new limit instance configured from {@code config} | ||
| */ | ||
| public static AimdLimit create(AimdLimitConfig config) { | ||
| return new AimdLimit(config); | ||
| } | ||
|
|
||
| /** | ||
| * Create a new instance customizing its configuration. | ||
| * | ||
| * @param consumer consumer of configuration builder | ||
| * @return a new limit instance configured from the builder | ||
| */ | ||
| public static AimdLimit create(Consumer<AimdLimitConfig.Builder> consumer) { | ||
| return builder() | ||
| .update(consumer) | ||
| .build(); | ||
| } | ||
|
|
||
| @Override | ||
| public <T> T invoke(Callable<T> callable) throws Exception { | ||
| return aimdLimitImpl.invoke(callable); | ||
| } | ||
|
|
||
| @Override | ||
| public void invoke(Runnable runnable) throws Exception { | ||
| aimdLimitImpl.invoke(runnable); | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<Token> tryAcquire(boolean wait) { | ||
| return aimdLimitImpl.tryAcquire(); | ||
| } | ||
|
|
||
| @SuppressWarnings("removal") | ||
| @Override | ||
| public Semaphore semaphore() { | ||
| return aimdLimitImpl.semaphore(); | ||
| } | ||
|
|
||
| @Override | ||
| public String name() { | ||
| return config.name(); | ||
| } | ||
|
|
||
| @Override | ||
| public String type() { | ||
| return TYPE; | ||
| } | ||
|
|
||
| @Override | ||
| public AimdLimitConfig prototype() { | ||
| return config; | ||
| } | ||
|
|
||
| @Override | ||
| public Limit copy() { | ||
| return config.build(); | ||
| } | ||
| } |
Oops, something went wrong.