uftrace script does not support python virtual environment

[kyet]

This issue is not intended to be a bug report or feature request. Instead, it is being shared with other users who may encounter a similar problem.

Therefore, feel free to close this issue without further action.

---

As shown in the log below, I am unable to import numpy in a virtual environment (venv) where numpy is installed.

(uftrace) (ubuntu) ~/ws/uftrace $ cat import_numpy.py
import numpy

(uftrace) (ubuntu) ~/ws/uftrace $ uftrace script -S import_numpy.py
Traceback (most recent call last):
  File "/home/kwangjin/ws/uftrace/import_numpy.py", line 1, in <module>
    import numpy
ModuleNotFoundError: No module named 'numpy'
WARN: "import_numpy.py" cannot be imported!
WARN: failed to init python scripting

(uftrace) (ubuntu) ~/ws/uftrace $ uv pip show numpy
Name: numpy
Version: 2.3.2
Location: /home/kwangjin/ws/uftrace/.venv/lib/python3.13/site-packages
Requires:
Required-by:

I have investigated this issue further.

The uftrace script uses the libpython3.xx.so (hereinafter referred to as libpython) library instead of the python3 binary. However, venv does not support libpython by design.

venvs don't contain copies of libpython or header files by design.
python/cpython#87500 (comment)

This limitation applies to the Python venv module (python -m venv), virtualenv, and uv venv. Conda, on the other hand, supports libpython. However, since most venvs do not support it, Conda's case is an exception.

The only provider that offers this is conda.
astral-sh/uv#6812 (comment)

Even if venv were to support libpython, the current structure of uftrace requires the libpython version to be determined at compile time. Therefore, switching to a different venv would require rebuilding uftrace. The relevant source code is as follows:

check-deps/Makefile.check

ifneq ($(wildcard $(objdir)/check-deps/have_libpython3),)
  ..
  COMMON_CFLAGS += -DLIBPYTHON_VERSION=$(shell pkg-config python3$(EMBED) --modversion)
..

uftrace uses pkg-config, which can be executed directly with the following result,

(uftrace) (ubuntu) ~/ws/uftrace $ python --version
Python 3.13.5
(uftrace) (ubuntu) ~/ws/uftrace $ pkg-config python3 --modversion
3.12

The LIBPYTHON_VERSION is used in the following code,

utils/script-python.c

static const char libpython[] = "libpython" stringify(LIBPYTHON_VERSION) ".so";
..
static int load_python_api_funcs(void)
{
	python_handle = dlopen(libpython, RTLD_LAZY | RTLD_GLOBAL);
    ..

The debug log shows: libpython3.12.so is loaded

(uftrace) (ubuntu) ~/ws/uftrace $ uftrace -v script -S import_numpy.py
uftrace: running uftrace v0.15 ( aarch64 dwarf python3 luajit tui perf sched dynamic kernel )
uftrace: reading uftrace.data/task.txt file
fstack: fixup for some special functions
script: script_init_for_python("import_numpy.py")
script: libpython3.12.so is loaded
Traceback (most recent call last):
  File "/home/kwangjin/ws/uftrace/import_numpy.py", line 1, in <module>
    import numpy
ModuleNotFoundError: No module named 'numpy'
WARN: "import_numpy.py" cannot be imported!
WARN: failed to init python scripting
script: script_finish_for_python()

In conclusion, the uftrace script uses libpython instead of python, and venv does not support libpython.

[honggyukim]

Thanks for the report. It looks it's not easy to fix for now but worth having the analysis report here!