Skip to content

Latest commit

 

History

History
141 lines (112 loc) · 4.51 KB

README.md

File metadata and controls

141 lines (112 loc) · 4.51 KB

SustainKieker Hackathon: Reverse Engineering of Research Software

🗓️ Thu 27/02, 13:30
📍 Room SR 206, Building 30.70, Campus South (KIT)
ℹ️ Abstract
🌐 The SustainKieker Project Homepage
🔬 The Kieker Observability Framework

What is it about?

Kieker observability framework features monitoring and analysis capabilities. OpenTelemetry, in comparison, provides means to monitor the program but analysis. Our new Otkt DSL can define a mapping from an OpenTelemetry Span to a Kieker record. Using Otkt, we show how we can use OpenTelemetry to collect monitoring data from a Python program and send it to a Kieker Analysis endpoint.

Prerequistes

  1. Python3

  2. Java (e.g., OpenJDK 21)

    # Fedora 40, 41, 42
sudo dnf install java-21-openjdk-devel
# Ubuntu 20.04, 22.04, 24.04, 24.10
sudo apt install openjdk-21-jdk

  3. Maven

    1. Install manually: https://maven.apache.org/install.html
    2. From distribution repositories: 
      # Fedora 40, 41, 42
sudo dnf install maven
# Ubuntu 20.04, 22.04, 24.04, 24.10
sudo apt install maven

  4. GNU plotutils

    # Fedora 40, 41, 42
sudo dnf install plotutils
# Ubuntu 20.04, 22.04, 24.04, 24.10
sudo apt install plotutils

  5. The graphviz graph visualization tools

    # Fedora 40, 41, 42
sudo dnf install graphviz
# Ubuntu 20.04, 22.04, 24.04, 24.10
sudo apt install graphviz

Important

Instrumenting a Python software may require downloading the source code and building a package. In this case, follow the instruction below and repackage it yourself. It could be as simple as running python3 -m build. Otherwise, you need to follow the provided build instruction.

# Instrument the target software before running the below commands
pip install build
python3 -m build

Otkt DSL

  • BUILD.md describes how to build an Otkt program.
  • The repository provides all artifacts from an Otkt build.

Instructions

  1. Build the Otkt collector with Maven.

    cd collector
mvn clean package

  2. To instrument a python program, export the parent path of the otkt directory to the PYTHONPATH variable:

    export PYTHONPATH=/path/to/parent/of/otkt:$PYTHONPATH

    i.e.,

    export PYTHONPATH=/path/to/hackathon/python:$PYTHONPATH

  3. Prepend the following line to all Python files of the target program:

    from otkt.otelinit import tracer

  4. Prepend the following line before all Python method definitions:

    @instrument
    • It is important to adhere to the indentation of each method definitions.

  5. Make sure you have all required dependencies below. A python project usually comes with requirements.txt. Append all dependencies in res/requirements.txt to the target software's requirements.txt.

    • If the software does not provide such file, you may copy the provided requirements.txt file and use it as followed by:
    pip install -r requirements.txt

  6. Run the Otkt collector on a separate terminal.

    java -jar /path/to/Collector-0.0.1-SNAPSHOT-jar-with-dependencies.jar -c /path/to/config.txt

  7. On a new terminal, run the target program (e.g., python3 main.py as below). You need the PYTHONPATH environmental variable exported to locate all otkt modules. See that the Otkt collector runs in the background to receive all created monitoring records. The collected monitoring recrods can be found in the output destination you specified in "config.txt" above.

    export PYTHONPATH=${PWD}:$PYTHONPATH
python3 main.py

  8. we use the Kieker Trace Analysis to analyze the target program.

Options

  1. The Otkt collector stores all received records under /tmp. E.g.,

    /tmp/kieker-20250217-181132-41826294550971-UTC--KIEKER/
/tmp/kieker-20250218-093839-12420615078112-UTC--KIEKER/
/tmp/kieker-20250218-114626-20087904425947-UTC--KIEKER/

    You can change the location by changing the value for kieker.monitoring.writer.filesystem.FileWriter.customStoragePath in res/config.txt.