Profiling Kratos with LineProfile

Profiling Kratos with LineProfile

This page details the steps to follow in order to profile Kratos with a python line profiler. Please be aware that this is an intrusive process, and you will have to modify some of your code.

Obtaining the profiler: You can find the python line profiler an all the relevant information here: https://github.com/pyutils/line_profiler.

If you prefer only to install it just:

python -m pip install line_profiler

Preparing your code. The python line profiler needs to know beforehand which functions are you interested in debugger. In that sense, it is helpful that you already have a slight idea on what sections of your code are interested on.

In order to mark such parts, you will have to use the @profile decorator. A example is presented below:

@profile def my_function(): var = calculate() ... etc ...

Unfortunately, this process will make your code break if you try to run it without the profiler. We are working on integrating a mechanism that solved this inside Kratos, but in the meantime we recommend you to use a launcher script to run your code. This will allow to ignore the decorators without removing them from the code if you intend to run the code normally.

Such launcher script have to look like this:

import os
import sys
import builtins
import line_profiler

prof = line_profiler.LineProfiler()

builtins.__dict__['profile'] = prof

exec(open(sys.argv[1]).read())

with open("profile_"+str(os.getpid())+".prof", "w+") as prof_file:
    prof.print_stats(prof_file)

print("Profile output written in:", "profile_"+str(os.getpid())+".prof")

We do recommend to launch your case with this launcher as well if you intend to run with MPI, as it will automatically generate a different report for every process. An example of an invocation call will look like this:

Serial

python profiler.py MainKratos.py

Distributed

mpirun -np [N] --output-filename /home/croig/cases/moldshape-1M/mpirun-mold-16 python profiler.py MainKratos.py