Can't link in /librcl_interfaces__rosidl_generator_py.so due to it not linking in libpython

[tony-p]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

I'm having an issue on linux (humble) that when it tries to link in the librcl_interfaces__rosidl_generator_py.so it is missing a number of symbols resulting in undefined reference symbols

env/lib/librcl_interfaces__rosidl_generator_py.so: undefined reference to `PyLong_AsSize_t'

It is essentially the same problem as this https://robotics.stackexchange.com/questions/25039/generated-rosidl-generator-py-so-files-issue

And it is the same for the robostack provided version, however the workarounds don't seem to work and I think it is because it is already too late and should have been applied to these packages or the ones in between.

$ ldd -r .pixi/env/lib/librcl_interfaces__rosidl_generator_py.so
	linux-vdso.so.1 (0x00007ffcd3fd2000)
	libbuiltin_interfaces__rosidl_generator_py.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so (0x00007f51715d8000)
	librcl_interfaces__rosidl_generator_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./librcl_interfaces__rosidl_generator_c.so (0x00007f51715be000)
	librosidl_runtime_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./librosidl_runtime_c.so (0x00007f51715b2000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5171200000)
	libbuiltin_interfaces__rosidl_generator_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/././libbuiltin_interfaces__rosidl_generator_c.so (0x00007f517159b000)
	librcutils.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/././librcutils.so (0x00007f5171584000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f51715ef000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f517157f000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f517157a000)
undefined symbol: PyObject_GetAttrString	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_New	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_Release	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetAttrString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_AsUTF8String	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBytes_FromStringAndSize	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_ToContiguous	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyExc_RuntimeError	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyErr_SetString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyFloat_FromDouble	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Size	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CheckBuffer	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBool_FromLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsSize_t	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_SetItem	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallFunctionObjArgs	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_Size	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_DecodeUTF8	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_TrueStruct	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetBuffer	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Fast	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)

Would patching the librcl_interfaces__rosidl_generator_py build to force link in libpython3 be the best solution here?

Installed packages

pixi Lock file is too long too paste here, will try find a more concise way to paste, but the relevant package is

- platform: linux-64
  name: ros-humble-rcl-interfaces
  version: 1.2.1
  category: main
  manager: conda
  dependencies:
  - libgcc-ng >=12
  - libgcc-ng >=12
  - libstdcxx-ng >=12
  - numpy >=1.21.6,<2.0a0
  - python
  - python_abi 3.10.* *_cp310
  - ros-humble-builtin-interfaces
  - ros-humble-ros-workspace
  - ros-humble-rosidl-default-runtime
  - ros2-distro-mutex 0.3.*
  - ros2-distro-mutex 0.3.* humble
  url: https://repo.prefix.dev/robostack-staging/linux-64/ros-humble-rcl-interfaces-1.2.1-py310h7c61026_3.tar.bz2
  hash:
    md5: 7123a8c0b2d39745f42ee72e444e8caf
    sha256: d66ad32b9a592a056b7b6a4eda2840a3b67d3b506a7ee3e29528bd81da9731c5
  build: py310h7c61026_3
  arch: x86_64
  subdir: linux-64
  build_number: 3
  size: 343519
  timestamp: 1675688249136

Environment info

pixi info
pixi 0.10.0

Platform            : linux-64
Virtual packages    : __unix=0=0
                    : __linux=6.2.0=0
                    : __glibc=2.35=0
                    : __archspec=1=x86_64
Cache dir           : /home/me/.cache/rattler/cache
Auth storage        : /home/me/.rattler/rattler_auth_store.json

Project
------------

Manifest file       : /home/me/workspace/project/pixi.toml
Dependency count    : 49
Last updated        : 19-12-2023 16:24:34
Target platforms    : win-64
                    : linux-64

[tony-p]

Update: force linking in the output of $(python3-config --ldflags --embed) (from the linked issue) in the failing package does work, although as a temporary fix for another linking issue I had added the env/lib in LD_LIBRARY_PATH and then it tried to link in the wrong version of 'libfreebl3.so` which required a newer glibc

[Tobias-Fischer]

Hi @tony-p - could you please try again with the recently built updated packages?

[tony-p]

Tried again very naively and same issue occurs however checked and is taking build 3 instead of 5 of rcl-interfaces 1.2.1 so will look into a bit more when I have a little more time

[tony-p]

Dug a little deeper, and now have the build 5 and end up with the same errors. Further the ldd shows that these still aren't linked (below)

Further I encountered the following issues:

• for some reason the dependency resolution was choosing an old version of sysroot that had libc v2.12. Forcing that sysroot >=2.17 solved it without other dependency conflicts so don't know why it wasn't chosen in the first place. This also means that there is the sysroot package and the virtual __glibc package both providing libc.
• many ament_*lint executables are in the lib/<tool> directory which results in them not being found by cmake as they are not on the path. Manually adding each tool to the path (temporarily) solved that issue

ldd -r .pixi/env/lib/librcl_interfaces__rosidl_generator_py.so
        linux-vdso.so.1 (0x00007ffc8c0c7000)
        libbuiltin_interfaces__rosidl_generator_py.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so (0x00007f375c73b000)
        librcl_interfaces__rosidl_generator_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./librcl_interfaces__rosidl_generator_c.so (0x00007f375c721000)
        librosidl_runtime_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./librosidl_runtime_c.so (0x00007f375c715000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f375c4e6000)
        libbuiltin_interfaces__rosidl_generator_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/././libbuiltin_interfaces__rosidl_generator_c.so (0x00007f375c4de000)
        librcutils.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/././librcutils.so (0x00007f375c4c7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f375c751000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f375c4c2000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f375c4bd000)
undefined symbol: PyObject_GetAttrString        (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString        (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc   (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLong       (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLong (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject   (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong       (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_New    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLongLong   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_Release      (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLongLong     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetAttrString        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_AsUTF8String        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBytes_FromStringAndSize     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_ToContiguous (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyExc_RuntimeError    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyErr_SetString       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyFloat_FromDouble    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Size       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CheckBuffer  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBool_FromLong       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsSize_t       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_SetItem        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallFunctionObjArgs  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_Size (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_DecodeUTF8  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_TrueStruct        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetBuffer    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLongLong     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLongLong   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Fast       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)

[Tobias-Fischer]

I wonder if this should be public:

ros-humble/patch/ros-humble-rosidl-generator-py.patch

Line 61 in 52756fd

+target_link_libraries(${_target_name_lib} PRIVATE Python3::NumPy Python3::Module)

Any ideas @traversaro @wolfv?

[Tobias-Fischer]

I've just pushed 7d0d8be which I have locally confirmed fixes this issue. If you rebuild ros-humble-rosidl-generator-py and then ros-humble-rcl-interfaces the issue will go away. Unfortunately at the moment I have not the capacity to build this in CI; but it would be easy to do (please see Contributing guidelines if you'd like to open a PR @tony-p).

[tony-p]

Great thanks, will try find some time to try soon (also limited capacity) and if fixes and have time will definitely look into the CI build

[EzraBrooks]

Hi, what's left to get this across the finish line? I'm pretty blocked by this issue and would be happy to help.

[tony-p]

Hi, what's left to get this across the finish line? I'm pretty blocked by this issue and would be happy to help.

I suggest to read the thread in this PR #169

In short the PR ended up reverting the change due to downstream build dependency issues. Wolf rewrote the cmake to more modern standards, but I haven't had the chance to rebuild try if that also fixed the issue (so that would be step 1).

Given that the python libraries are still not public, I fear the issue may still be there.

The issue for the downstream builds was that they failed to find python as dependency (as opposed to missing symbols), One (hacky?) solution to that was to include rosidl_generator as a dependency purely as it has a additional exported cmake file that includes finding the python dependencies (

ros-humble/patch/ros-humble-rosidl-generator-py.patch

Line 13 in 71d4b56

+find_package(Python3 REQUIRED COMPONENTS Development NumPy)

).

So if it doesn't work with the updates from Wolf, when the interface pacakges are themselves a dependency, they need to be able to give this information through, ideally in a cleaner way then adding another dependency that happens to do that.

As a short term hack fix in your own builds that are causing issues you can add something like the following to force the un found packages to be linked

if(UNIX)
  # Hack to link in missing/can't find libraries on linux
  execute_process(COMMAND bash "-c" "echo $(python3-config --ldflags --embed)" OUTPUT_VARIABLE BUILD_PYTHON_LINK_FLAGS)
  separate_arguments(FORCE_LINK_FLAGS UNIX_COMMAND ${BUILD_PYTHON_LINK_FLAGS})
  list(APPEND FORCE_LINK_FLAGS
  -lstd_msgs__rosidl_generator_py
  )
  add_link_options(${FORCE_LINK_FLAGS})
endif()

[EzraBrooks]

Can you clarify what you mean by "include rosidl_generator as a dependency"? Do you mean as a package.xml dependency, an ament_cmake dependency, a pure CMake dependency, or a source build within the same workspace - or some combination of those?

Just adding it as a source build in my workspace doesn't seem to help (I assume because the other packages don't have direct dependencies on it, so it's not resolved to be built first)