Skip to content

Commit d80a7df

Browse files
anupama-pathirageanupama-pathirage
authored
Merge pull request #125 from wso2/capacity_planning
Add capacity planning section and related docs
2 parents 5b1816b + 07a3ce7 commit d80a7df

File tree

4 files changed

+165
-1
lines changed

4 files changed

+165
-1
lines changed

‎en/docs/deploy/capacity-planning/overview.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Capacity Planning for WSO2 Integrator: BI
2+
This section provides an overview of the performance characteristics and capacity planning considerations for WSO2 Integrator: BI when deployed in a cloud environment. The performance reports linked below detail the results of various integration scenarios under specific load conditions.
3+
4+
The tests were conducted on the following AWS EC2 instance configuration:
5+
6+
| Description | Value |
7+
| :---- | :---- |
8+
| EC2 Instance type | c5.large |
9+
| CPUs | 2 |
10+
| Thread(s) per core | 2 |
11+
| Core(s) per socket | 1 |
12+
| Socket(s) | 1 |
13+
| Processor model name | Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz |
14+
| System Memory | 4 GiB |
15+
| Storage | 8G nvme0n1 |
16+
| Operating System | Ubuntu 18.04.6 LTS |
17+
| Kernel version | 5.4.0-1094-aws |
18+
19+
## Performance reports
20+
21+
Explore the detailed performance reports for various integration types:
22+
23+
* [Performance report for REST Services (HTTP and HTTPS)](/deploy/capacity-planning/performance-report-for-rest-services)
24+

‎en/docs/deploy/capacity-planning/performance-report-for-rest-services.md‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Performance report for REST services (HTTP and HTTPS)
2+
3+
## Overview
4+
5+
| Test Scenarios | Description |
6+
| --- | --- |
7+
| Passthrough HTTP service (h1c -> h1c) | An HTTP Service, which forwards all requests to an HTTP back-end service. |
8+
| Passthrough HTTPS service (h1 -> h1) | An HTTPS Service, which forwards all requests to an HTTPS back-end service. |
9+
| JSON to XML transformation HTTP service | An HTTP Service, which transforms JSON requests to XML and then forwards all requests to an HTTP back-end service. |
10+
| JSON to XML transformation HTTPS service | An HTTPS Service, which transforms JSON requests to XML and then forwards all requests to an HTTPS back-end service. |
11+
| Passthrough HTTP/2(over TLS) service (h2 -> h2) | An HTTPS Service exposed over HTTP/2 protocol, which forwards all requests to an HTTP/2(over TLS) back-end service. |
12+
| Passthrough HTTP/2(over TLS) service (h2 -> h1) | An HTTPS Service exposed over HTTP/2 protocol, which forwards all requests to an HTTPS back-end service. |
13+
| Passthrough HTTP/2(over TLS) service (h2 -> h1c) | An HTTPS Service exposed over HTTP/2 protocol, which forwards all requests to an HTTP back-end service. |
14+
| HTTP/2 client and server downgrade service (h2 -> h2) | An HTTP/2(with TLS) server accepts requests from an HTTP/1.1(with TLS) client and the HTTP/2(with TLS) client sends requests to an HTTP/1.1(with TLS) back-end service. Both the upstream and downstream connections are downgraded to HTTP/1.1(with TLS). |
15+
16+
Our test client is [Apache JMeter](https://jmeter.apache.org/index.html). We test each scenario for a fixed duration of
17+
time. We split the test results into warm-up and measurement parts and use the measurement part to compute the
18+
performance metrics.
19+
20+
A majority of test scenarios use a [Netty](https://netty.io/) based back-end service, which echoes back any request
21+
posted to it after a specified period of time.
22+
23+
!!! note
24+
The code and instructions for running these tests are provided [here](https://github.com/ballerina-platform/ballerina-performance).
25+
26+
## Performance metrics
27+
28+
We run the performance tests under different numbers of concurrent users, message sizes (payloads), and back-end service
29+
delays.
30+
31+
The main performance metrics:
32+
33+
1. **Throughput**: The number of requests that the Ballerina service processes during a specific time interval (e.g., per second).
34+
2. **Response Time**: The end-to-end latency for an operation of invoking a Ballerina service. The complete distribution of response times was recorded.
35+
36+
In addition to the above metrics, we measure the load average and several memory-related metrics.
37+
38+
## Test parameters
39+
40+
The following are the test parameters.
41+
42+
| Test Parameter | Description | Values |
43+
| --- | --- | --- |
44+
| Scenario Name | The name of the test scenario. | Refer to the above table. |
45+
| Heap Size | The amount of memory allocated to the application | 2G |
46+
| Concurrent Users | The number of users accessing the application at the same time. | 100, 200, 500, 1000 |
47+
| Message Size (Bytes) | The request payload size in Bytes. | 500, 1000, 10000 |
48+
| Back-end Delay (ms) | The delay added by the back-end service. | 0 |
49+
50+
The duration of each test is **1200 seconds**. The warm-up period is **600 seconds**.
51+
The measurement results are collected after the warm-up period.
52+
53+
A [**c5.large** Amazon EC2 instance](https://aws.amazon.com/ec2/instance-types/) was used to install Ballerina.
54+
55+
The following are the measurements collected from each performance test conducted for a given combination of
56+
test parameters.
57+
58+
| Measurement | Description |
59+
| --- | --- |
60+
| Error % | Percentage of requests with errors |
61+
| Average Response Time (ms) | The average response time of a set of results |
62+
| Standard Deviation of Response Time (ms) | The “Standard Deviation” of the response time. |
63+
| 99th Percentile of Response Time (ms) | 99% of the requests took no more than this time. The remaining samples took at least as long as this |
64+
| Throughput (Requests/sec) | The throughput measured in requests per second. |
65+
| Average Memory Footprint After Full GC (M) | The average memory consumed by the application after a full garbage collection event. |
66+
67+
## Test results
68+
69+
The following is the summary of performance test results collected for the measurement period.
70+
71+
| Scenario Name | Concurrent Users | Message Size (Bytes) | Back-end Service Delay (ms) | Error % | Throughput (Requests/sec) | Average Response Time (ms) | Standard Deviation of Response Time (ms) | 99th Percentile of Response Time (ms)
72+
|---|---:|---:|---:|---:|---:|---:|---:|---:|
73+
| Passthrough HTTP service (h1c -> h1c) | 100 | 500 | 0 | 0 | 8935.69 | 11.13 | 4.78 | 26 |
74+
| Passthrough HTTP service (h1c -> h1c) | 100 | 1000 | 0 | 0 | 9341.58 | 10.65 | 4.97 | 26 |
75+
| Passthrough HTTP service (h1c -> h1c) | 100 | 10000 | 0 | 0 | 6358.34 | 15.66 | 6.52 | 36 |
76+
| Passthrough HTTP service (h1c -> h1c) | 200 | 500 | 0 | 0 | 10719.1 | 18.59 | 6.93 | 37 |
77+
| Passthrough HTTP service (h1c -> h1c) | 200 | 1000 | 0 | 0 | 9108.9 | 21.89 | 7.25 | 43 |
78+
| Passthrough HTTP service (h1c -> h1c) | 200 | 10000 | 0 | 0.01 | 6348.51 | 31.42 | 134.77 | 78 |
79+
| Passthrough HTTP service (h1c -> h1c) | 500 | 500 | 0 | 0 | 8706.5 | 57.35 | 15.55 | 96 |
80+
| Passthrough HTTP service (h1c -> h1c) | 500 | 1000 | 0 | 0 | 8946.24 | 55.81 | 14.4 | 92 |
81+
| Passthrough HTTP service (h1c -> h1c) | 500 | 10000 | 0 | 0.01 | 6393.06 | 78.11 | 25.5 | 145 |
82+
| Passthrough HTTP service (h1c -> h1c) | 1000 | 500 | 0 | 0 | 8665.7 | 115.29 | 24.02 | 172 |
83+
| Passthrough HTTP service (h1c -> h1c) | 1000 | 1000 | 0 | 0 | 8944.47 | 111.69 | 19.91 | 165 |
84+
| Passthrough HTTP service (h1c -> h1c) | 1000 | 10000 | 0 | 0.01 | 6296.27 | 158.68 | 41.14 | 263 |
85+
| JSON to XML transformation HTTP service | 100 | 500 | 0 | 0 | 1590.82 | 62.8 | 252.15 | 138 |
86+
| JSON to XML transformation HTTP service | 100 | 1000 | 0 | 0 | 998.79 | 100.05 | 379 | 188 |
87+
| JSON to XML transformation HTTP service | 100 | 10000 | 0 | 0 | 84.2 | 1186.62 | 521.64 | 1903 |
88+
| JSON to XML transformation HTTP service | 200 | 500 | 0 | 0 | 1498.83 | 133.35 | 180.62 | 295 |
89+
| JSON to XML transformation HTTP service | 200 | 1000 | 0 | 0.02 | 1001.07 | 199.77 | 693.66 | 397 |
90+
| JSON to XML transformation HTTP service | 200 | 10000 | 0 | 0 | 70.63 | 2823.67 | 1185.64 | 4479 |
91+
| JSON to XML transformation HTTP service | 500 | 500 | 0 | 0 | 1478.51 | 338.23 | 237.25 | 747 |
92+
| JSON to XML transformation HTTP service | 500 | 1000 | 0 | 0 | 905.56 | 552.17 | 340.06 | 1175 |
93+
| JSON to XML transformation HTTP service | 500 | 10000 | 0 | 99.16 | 15.89 | 29911.92 | 1485.55 | 31103 |
94+
| JSON to XML transformation HTTP service | 1000 | 500 | 0 | 0 | 1445.42 | 691.67 | 373.82 | 1551 |
95+
| JSON to XML transformation HTTP service | 1000 | 1000 | 0 | 0 | 885.2 | 1128.63 | 1467.77 | 2239 |
96+
| JSON to XML transformation HTTP service | 1000 | 10000 | 0 | 0 | 59.63 | 16479.55 | 5309.9 | 24959 |
97+
| Passthrough HTTPS service (h1 -> h1) | 100 | 500 | 0 | 0 | 8282.45 | 12.02 | 5.67 | 28 |
98+
| Passthrough HTTPS service (h1 -> h1) | 100 | 1000 | 0 | 0 | 7676.52 | 12.97 | 6.39 | 31 |
99+
| Passthrough HTTPS service (h1 -> h1) | 100 | 10000 | 0 | 0 | 4405.6 | 22.63 | 38.69 | 70 |
100+
| Passthrough HTTPS service (h1 -> h1) | 200 | 500 | 0 | 0 | 8298.14 | 24.04 | 9.49 | 53 |
101+
| Passthrough HTTPS service (h1 -> h1) | 200 | 1000 | 0 | 0 | 8185.02 | 24.37 | 9.6 | 53 |
102+
| Passthrough HTTPS service (h1 -> h1) | 200 | 10000 | 0 | 0.01 | 4314.93 | 46.26 | 12.9 | 97 |
103+
| Passthrough HTTPS service (h1 -> h1) | 500 | 500 | 0 | 0 | 7631.44 | 65.43 | 18.17 | 111 |
104+
| Passthrough HTTPS service (h1 -> h1) | 500 | 1000 | 0 | 0 | 7322.7 | 68.2 | 18.11 | 114 |
105+
| Passthrough HTTPS service (h1 -> h1) | 500 | 10000 | 0 | 0.01 | 4187 | 119.3 | 29.12 | 226 |
106+
| Passthrough HTTPS service (h1 -> h1) | 1000 | 500 | 0 | 0 | 7257.55 | 137.64 | 30.77 | 207 |
107+
| Passthrough HTTPS service (h1 -> h1) | 1000 | 1000 | 0 | 0 | 7213.92 | 138.47 | 30.72 | 209 |
108+
| Passthrough HTTPS service (h1 -> h1) | 1000 | 10000 | 0 | 0.01 | 4076.18 | 245.27 | 59.83 | 439 |
109+
| JSON to XML transformation HTTPS service | 100 | 500 | 0 | 0 | 1507.24 | 66.28 | 337.13 | 158 |
110+
| JSON to XML transformation HTTPS service | 100 | 1000 | 0 | 0 | 967.15 | 103.32 | 311.01 | 206 |
111+
| JSON to XML transformation HTTPS service | 100 | 10000 | 0 | 2 | 83.65 | 1181 | 4156.47 | 30079 |
112+
| JSON to XML transformation HTTPS service | 200 | 500 | 0 | 0 | 1380.33 | 144.81 | 192.98 | 349 |
113+
| JSON to XML transformation HTTPS service | 200 | 1000 | 0 | 0.04 | 897.78 | 222.73 | 1027.82 | 501 |
114+
| JSON to XML transformation HTTPS service | 200 | 10000 | 0 | 2.27 | 76.57 | 2602.91 | 4886.23 | 30079 |
115+
| JSON to XML transformation HTTPS service | 500 | 500 | 0 | 0 | 1505.97 | 332.02 | 446.28 | 1663 |
116+
| JSON to XML transformation HTTPS service | 500 | 1000 | 0 | 0 | 881.78 | 566.83 | 1394.24 | 1159 |
117+
| JSON to XML transformation HTTPS service | 500 | 10000 | 0 | 100 | 21.33 | 22693.11 | 11309.03 | 42495 |
118+
| JSON to XML transformation HTTPS service | 1000 | 500 | 0 | 0 | 1424.83 | 701.47 | 591.2 | 3487 |
119+
| JSON to XML transformation HTTPS service | 1000 | 1000 | 0 | 0 | 806.11 | 1239.02 | 1167.47 | 3215 |
120+
| JSON to XML transformation HTTPS service | 1000 | 10000 | 0 | 0 | 61.55 | 15956.62 | 4774.51 | 24319 |
121+
| HTTP/2 client and server downgrade service (h2 -> h2) | 100 | 500 | 0 | 0 | 8289.32 | 12.01 | 5.58 | 28 |
122+
| HTTP/2 client and server downgrade service (h2 -> h2) | 100 | 1000 | 0 | 0 | 7675.18 | 12.97 | 6.07 | 30 |
123+
| HTTP/2 client and server downgrade service (h2 -> h2) | 100 | 10000 | 0 | 0 | 4327.11 | 23.03 | 6.94 | 49 |
124+
| HTTP/2 client and server downgrade service (h2 -> h2) | 200 | 500 | 0 | 0 | 7626.89 | 26.16 | 8.86 | 52 |
125+
| HTTP/2 client and server downgrade service (h2 -> h2) | 200 | 1000 | 0 | 0 | 8012.4 | 24.89 | 9.59 | 54 |
126+
| HTTP/2 client and server downgrade service (h2 -> h2) | 200 | 10000 | 0 | 0.04 | 4206.28 | 47.46 | 13.18 | 100 |
127+
| HTTP/2 client and server downgrade service (h2 -> h2) | 500 | 500 | 0 | 0 | 7562.06 | 66.03 | 18.66 | 113 |
128+
| HTTP/2 client and server downgrade service (h2 -> h2) | 500 | 1000 | 0 | 0 | 7174.09 | 69.61 | 17.7 | 114 |
129+
| HTTP/2 client and server downgrade service (h2 -> h2) | 500 | 10000 | 0 | 0 | 4113.17 | 121.44 | 33.72 | 253 |
130+
| HTTP/2 client and server downgrade service (h2 -> h2) | 1000 | 500 | 0 | 0 | 7462.01 | 133.86 | 36.73 | 218 |
131+
| HTTP/2 client and server downgrade service (h2 -> h2) | 1000 | 1000 | 0 | 0 | 7202.02 | 138.7 | 35.71 | 221 |
132+
| HTTP/2 client and server downgrade service (h2 -> h2) | 1000 | 10000 | 0 | 0.01 | 4017.91 | 248.82 | 61.31 | 447 |
133+
134+
135+
!!! note
136+
Ballerina Swan Lake 2201.12.4 was used for testing with default configurations. Your results may vary depending on the Ballerina version and configuration used.

‎en/docs/index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,8 @@
5353
{"name": "Deploy to Devant", "url": "deploy/deploy-to-devant/"},
5454
{"name": "Containerized Deployment", "url": "deploy/containerized-deployment/overview"},
5555
{"name": "VM-based Deployment", "url": "deploy/vm-based-deployment/overview"},
56-
{"name": "Managing Configurations", "url": "deploy/managing-configurations"}
56+
{"name": "Managing Configurations", "url": "deploy/managing-configurations"},
57+
{"name": "Capacity Planning", "url": "deploy/capacity-planning/overview"}
5758
]
5859
}
5960
],

‎en/mkdocs.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,9 @@ nav:
144144
- Decentralized Deployment: deploy/vm-based-deployment/de-centralized-deployment.md
145145
- GitHub Action for CICD: deploy/vm-based-deployment/github-action-for-cicd.md
146146
- Managing configurations: deploy/managing-configurations.md
147+
- Capacity Planning:
148+
- Overview: deploy/capacity-planning/overview.md
149+
- Performance Report for REST Services: deploy/capacity-planning/performance-report-for-rest-services.md
147150
- Observability and Monitoring:
148151
- Overview: observability-and-monitoring/overview.md
149152
- "Monitoring with WSO2 Integrator: ICP": observability-and-monitoring/monitoring-with-wso2-integrator-icp.md

0 commit comments

Comments
 (0)