Skip to content

docs: document how to deal with dynamic linking #1022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@


intersphinx_mapping = {
"cmake": ("https://cmake.org/cmake/help/latest/", None),
"python": ("https://docs.python.org/3", None),
"packaging": ("https://packaging.pypa.io/en/stable", None),
"setuptools": ("https://setuptools.pypa.io/en/latest", None),
Expand All @@ -99,7 +100,7 @@
val[0]
for key, val in intersphinx_mapping.items()
# Only works with RTD hosted intersphinx
if key not in ("hatchling", "python")
if key not in ("hatchling", "python", "cmake")
]

nitpick_ignore = [
Expand Down
75 changes: 75 additions & 0 deletions docs/guide/dynamic_link.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Dynamic linking

If you want to support dynamic linkages between python projects or system
libraries, you will likely encounter some issues in making sure the compiled
libraries/python bindings work after the wheel is created and the python project
is installed on the system. The most common issue are the missing hints pointing
to where the runtime libraries are located, specifically `RPATH` on Linux and
MacOS systems, and `PATH`/`os.add_dll_directory` on Windows systems. Here are
some recommendations on how to address them.

## Link to the static libraries

The easiest solution is to make sure you link to the static libraries
counterparts during the CMake build. How to achieve this depends on the specific
dependency, how it is imported in the CMake project and how the dependency is
packaged in each ecosystem.

For example for [Boost][FindBoost] this is controlled via the variable
`Boost_USE_STATIC_LIBS`.

[FindBoost]: inv:cmake:cmake:module#module:FindBoost

## Wheel repair tools

There are wheel repair tools for each operating system that bundle any dynamic
libraries used and patch the libraries/python bindings to point to prioritize
those libraries. The most common tools for these are [auditwheel] for Linux,
[delocate] for MacOS and [delvewheel] for Windows. [cibuildwheel] incorporates
these tools in its [repair wheel] feature.

These tools also rename the library with a unique hash to avoid any potential
name collision if the same library is being bundled by a different package, and
check if the packages confirm to standards like [PEP600] (`manylinux_X_Y`).
These tools do not allow to have cross wheel library dependency.

## Manual patching

You can manually make a relative RPath. This has the benefit of working when not
running scikit-build-core, as well.

The `RPATH` patching can be done as

```cmake
if(APPLE)
set(origin_token "@loader_path")
else()
set(origin_token "$ORIGIN")
endif()
set_property(TARGET <target> PROPERTY INSTALL_RPATH
"${origin_token}/install_path/to/dynamic_library"
)
```

For Windows patching, this has to be done at the python files using
`os.add_dll_directory` at the top-most package `__init__.py` file or top-level
python module files.

```python
import os
from pathlib import Path

dependency_dll_path = Path(__file__).parent / "install_path/to/dynamic_library"
os.add_dll_directory(str(dependency_dll_path))
```

<!-- prettier-ignore-start -->

[auditwheel]: https://pypi.org/project/auditwheel/
[delocate]: https://pypi.org/project/delocate/
[delvewheel]: https://pypi.org/project/delvewheel/
[cibuildwheel]: https://cibuildwheel.pypa.io/en/stable/
[repair wheel]: https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command
[PEP600]: https://peps.python.org/pep-0600

<!-- prettier-ignore-end -->
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ first Friday of every month at the same time. Our past meeting minutes are

guide/getting_started
guide/cmakelists
guide/dynamic_link
guide/crosscompile
guide/migration_guide
guide/build
Expand Down
Loading