Skip to content

Commit

Permalink
Update wording: MooBench measures all overhead of all pillars of obse…
Browse files Browse the repository at this point in the history
…rvability
  • Loading branch information
DaGeRe committed Aug 22, 2024
1 parent 1047246 commit 514d24b
Showing 1 changed file with 11 additions and 7 deletions.
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,35 @@
# The MooBench Monitoring Overhead Micro-Benchmark
# The MooBench Observability Overhead Micro-Benchmark

The MooBench micro-benchmarks can be used to quantify the performance overhead caused by monitoring framework components and different monitoring frameworks.
The MooBench micro-benchmarks can be used to quantify the performance overhead caused by observability framework components and different observability frameworks. Observability is achieved through its three pillars:
- Logs, i.e., timestamped information about system events,
- Metrics, i.e., numerical measurements of system behaviour, and
- Traces, i.e., representations of request, transaction or operation executions.
MooBench can measure the overhead that is created by obtaining any of these three pillars of observability from program execution.

Continuous measurement results are available here:
* Kiel University Server (Intel Xeon CPU E5620 @ 2.40 GHz, Debian 12): https://kieker-monitoring.net/performance-benchmarks/
* GH Actions Runner (Ubuntu 22.04[^1]): https://kieker-monitoring.github.io/moobench/dev/bench/

[^1]: According to https://docs.github.com/de/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners

Currenly (fully) supported monitoring frameworks are:
Currenly (fully) supported observability frameworks are:
* Kieker with Java (http://kieker-monitoring.net)
* OpenTelemetry with Java (https://opentelemetry.io/)
* inspectIT with Java (https://inspectit.rocks/)
For all combinations of supported monitoring frameworks $FRAMEWORK and languages $LANGUAGE, the folder frameworks contains a folder $FRAMEWORK-$LANGUAGE.
For all combinations of supported observability frameworks $FRAMEWORK and languages $LANGUAGE, the folder frameworks contains a folder $FRAMEWORK-$LANGUAGE.

## Approach

MooBenchs measures the overhead of monitoring by executing an example workload using different monitoring configurations, including *no instrumentation* (and hence no monitoring) at all, and full monitoring and data serialization via *binary writer*. The example workload consists of `$RECURSION_DEPTH` recursive calls of a function to itself. For example, the following graph shows the execution of MooBench in the *no instrumentation* configuration:
MooBenchs measures the overhead of gathering observability data by executing an example workload using different configurations, including *no instrumentation* (and hence no data gathering) at all, full distributed tracing and data serialization via *binary writer*. The example workload consists of `$RECURSION_DEPTH` recursive calls of a function to itself. For example, the following graph shows the execution of MooBench in the *no instrumentation* configuration:

```mermaid
graph TD;
BenchmarkingThreadNano.run-->MonitoredClassSimple.monitoredMethod;
BenchmarkingThreadNano.run-->MonitoredClassSimple.monitoredMethod;
MonitoredClassSimple.monitoredMethod-->MonitoredClassSimple.monitoredMethod;
MonitoredClassSimple.monitoredMethod-->id["Busy Wait"]
```

The *binary writer* configuration on the other hand includes the probe code, that is injected by the monitoring tool before and after the operation. For the Kieker monitoring framework, the probe inserts records into the `WriterController.writerQueue`, and these are then processed for finally writing binary data to the hard disk.
The *binary writer* configuration on the other hand includes the probe code, that is injected by the observability tool before and after the operation. For the Kieker monitoring framework, the probe inserts records into the `WriterController.writerQueue`, and these are then processed for finally writing binary data to the hard disk.

```mermaid
flowchart TD;
Expand Down

0 comments on commit 514d24b

Please sign in to comment.