Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small Improvements #5

Merged
merged 6 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v4.6.0
hooks:
- id: check-yaml
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.3.4
rev: v0.4.9
hooks:
- id: ruff-format
- id: ruff
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.9.0
rev: v1.10.0
hooks:
# note: mypy runs in an isolated environment and so has no access to third party packages
- id: mypy
entry: mypy src/maturin_import_hook/ tests/test_import_hook tests/runner.py
pass_filenames: false
additional_dependencies: ["pytest"]
- repo: https://github.com/codespell-project/codespell
rev: v2.2.6
rev: v2.3.0
hooks:
- id: codespell
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.39.0
rev: v0.41.0
hooks:
- id: markdownlint-fix
2 changes: 1 addition & 1 deletion Code-of-Conduct.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
education, socio-economic status, nationality, personal appearance, race,
education, socioeconomic status, nationality, personal appearance, race,
religion, or sexual identity and orientation.

## Our Standards
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ components take effect automatically like changes to python components do.
The import hook also provides conveniences such as
[importlib.reload()](https://docs.python.org/3/library/importlib.html#importlib.reload) support for maturin projects.

Documentation can be found at [maturin.rs](https://www.maturin.rs/import_hook).

## Usage

After installing `maturin`, install the import hook into a python virtual environment with:
After installing [maturin](https://www.maturin.rs/installation), install the import hook into a python virtual
environment with:

```shell
pip install maturin_import_hook
Expand All @@ -35,7 +38,7 @@ maturin_import_hook.install()

# when a rust package that is installed in editable mode is imported,
# that package will be automatically recompiled if necessary.
import pyo3_pure
import example_maturin_package

# when a .rs file is imported a project will be created for it in the
# maturin build cache and the resulting library will be loaded.
Expand Down
126 changes: 1 addition & 125 deletions docs/basics.md
Original file line number Diff line number Diff line change
@@ -1,127 +1,3 @@
# Basics

`maturin_import_hook` is a package that provides the capability for python `import` statements to trigger a rebuild
when importing a maturin project that is installed in editable mode (eg with `maturin develop` or `pip install -e`).
This makes development much more convenient as it brings the workflow of
developing Rust modules closer to the workflow of developing regular python modules.

The hook supports importing editable-installed pure Rust and mixed Rust/Python project
layouts as well as importing standalone `.rs` files.

## Installation

Install into a virtual environment then install so that the import hook is always active.

```shell
pip install maturin_import_hook
python -m maturin_import_hook site install # install into the active environment
```

Alternatively, instead of using `site install`, put calls to `maturin_import_hook.install()` into any script where you
want to use the import hook.

## Usage

```python
import maturin_import_hook

# install the import hook with default settings.
# this must be called before importing any maturin project
maturin_import_hook.install()

# when a rust package that is installed in editable mode is imported,
# that package will be automatically recompiled if necessary.
import pyo3_pure

# when a .rs file is imported a project will be created for it in the
# maturin build cache and the resulting library will be loaded.
#
# assuming subpackage/my_rust_script.rs defines a pyo3 module:
import subpackage.my_rust_script
```

The maturin project importer and the rust file importer can be used separately

```python
from maturin_import_hook import rust_file_importer
rust_file_importer.install()

from maturin_import_hook import project_importer
project_importer.install()
```

The import hook can be configured to control its behaviour

```python
import maturin_import_hook
from maturin_import_hook.settings import MaturinSettings

maturin_import_hook.install(
enable_project_importer=True,
enable_rs_file_importer=True,
settings=MaturinSettings(
release=True,
strip=True,
# ...
),
show_warnings=True,
# ...
)
```

Since the import hook is intended for use in development environments and not for
production environments, it may be a good idea to put the call to `maturin_import_hook.install()`
into `site-packages/sitecustomize.py` of your development virtual environment
([documentation](https://docs.python.org/3/library/site.html)). This will
enable the hook for every script run by that interpreter without calling `maturin_import_hook.install()`
in every script, meaning the scripts do not need alteration before deployment.

Installation into `sitecustomize.py` can be managed with the import hook cli using
`python -m maturin_import_hook site install`. The CLI can also manage uninstallation.

## CLI

The package provides a CLI interface for getting information such as the location and size of the build cache and
managing the installation into `sitecustomize.py`. For details, run:

```shell
python -m maturin_import_hook --help
```

## Environment Variables

The import hook can be disabled by setting `MATURIN_IMPORT_HOOK_ENABLED=0`. This can be used to disable
the import hook in production if you want to leave calls to `import_hook.install()` in place.

Build files will be stored in an appropriate place for the current system but can be overridden
by setting `MATURIN_BUILD_DIR`. These files can be deleted without causing any issues (unless a build is in progress).
The precedence for storing build files is:

- `MATURIN_BUILD_DIR`
- `<virtualenv_dir>/maturin_build_cache`
- `<system_cache_dir>/maturin_build_cache`
- e.g. `~/.cache/maturin_build_cache` on POSIX

See the location being used with the CLI: `python -m maturin_import_hook cache info`

## Logging

By default the `maturin_import_hook` logger does not propagate to the root logger. This is so that `INFO` level messages
are shown to the user without them having to configure logging (`INFO` level is normally not visible). The import hook
also has extensive `DEBUG` level logging that generally would be more noise than useful. So by not propagating, `DEBUG`
messages from the import hook are not shown even if the root logger has `DEBUG` level visible.

If you prefer, `maturin_import_hook.reset_logger()` can be called to undo the default configuration and propagate
the messages as normal.

When debugging issues with the import hook, you should first call `reset_logger()` then configure the root logger
to show `DEBUG` messages. You can also run with the environment variable `RUST_LOG=maturin=debug` to get more
information from maturin.

```python
import logging
logging.basicConfig(format='%(name)s [%(levelname)s] %(message)s', level=logging.DEBUG)
import maturin_import_hook
maturin_import_hook.reset_logger()
maturin_import_hook.install()
```
See [maturin.rs](https://maturin.rs/import_hook).
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "maturin-import-hook"
description = "import hook to load rust projects built with maturin"
description = "Import hook to load rust projects built with maturin"
authors = [
{name = "Matthew Broadway", email = "[email protected]"}
]
Expand Down
79 changes: 62 additions & 17 deletions src/maturin_import_hook/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
import json
import platform
import shutil
import site
import subprocess
from pathlib import Path

from maturin_import_hook._building import get_default_build_dir
from maturin_import_hook._site import (
get_sitecustomize_path,
get_usercustomize_path,
has_automatic_installation,
insert_automatic_installation,
remove_automatic_installation,
Expand All @@ -23,12 +25,12 @@ def _action_version(format_name: str) -> None:

try:
maturin_version = subprocess.check_output(["maturin", "--version"]).decode().strip()
except subprocess.CalledProcessError:
except (subprocess.CalledProcessError, FileNotFoundError):
maturin_version = "?"

try:
rustc_version = subprocess.check_output(["rustc", "--version"]).decode().strip()
except subprocess.CalledProcessError:
except (subprocess.CalledProcessError, FileNotFoundError):
rustc_version = "?"

try:
Expand Down Expand Up @@ -80,24 +82,30 @@ def _action_cache_clear(interactive: bool) -> None:

def _action_site_info(format_name: str) -> None:
sitecustomize_path = get_sitecustomize_path()
usercustomize_path = get_usercustomize_path()

_print_info(
{
"has_sitecustomize": sitecustomize_path.exists(),
"import_hook_installed": has_automatic_installation(sitecustomize_path),
"sitecustomize_path": str(sitecustomize_path),
"sitecustomize_exists": sitecustomize_path.exists(),
"sitecustomize_import_hook_installed": has_automatic_installation(sitecustomize_path),
"user_site_enabled": str(site.ENABLE_USER_SITE),
"usercustomize_path": str(usercustomize_path),
"usercustomize_exists": usercustomize_path.exists(),
"usercustomize_import_hook_installed": has_automatic_installation(usercustomize_path),
},
format_name,
)


def _action_site_install() -> None:
sitecustomize_path = get_sitecustomize_path()
insert_automatic_installation(sitecustomize_path)
def _action_site_install(*, user: bool, preset_name: str, force: bool) -> None:
module_path = get_usercustomize_path() if user else get_sitecustomize_path()
insert_automatic_installation(module_path, preset_name, force)


def _action_site_uninstall() -> None:
sitecustomize_path = get_sitecustomize_path()
remove_automatic_installation(sitecustomize_path)
def _action_site_uninstall(*, user: bool) -> None:
module_path = get_usercustomize_path() if user else get_sitecustomize_path()
remove_automatic_installation(module_path)


def _ask_yes_no(question: str) -> bool:
Expand Down Expand Up @@ -148,19 +156,56 @@ def _main() -> None:

site_action = subparsers.add_parser(
"site",
help="manage installation of the import hook into site-packages/sitecustomize.py (so it starts automatically)",
help=(
"manage installation of the import hook into site-packages/sitecustomize.py "
"or usercustomize.py (so it starts automatically)"
),
)
site_sub_actions = site_action.add_subparsers(dest="sub_action")
site_info = site_sub_actions.add_parser(
"info", help="information about the current status of installation into sitecustomize"
"info", help="information about the current status of installation into sitecustomize/usercustomize"
)
site_info.add_argument(
"-f", "--format", choices=["text", "json"], default="text", help="the format to output the data in"
)
site_sub_actions.add_parser(
"install", help="install the import hook into site-packages/sitecustomize.py so that it starts automatically"

install = site_sub_actions.add_parser(
"install",
help=(
"install the import hook into site-packages/sitecustomize.py "
"or usercustomize.py so that it starts automatically"
),
)
install.add_argument(
"-f",
"--force",
action="store_true",
help="whether to overwrite any existing managed import hook installation",
)
install.add_argument(
"--preset",
default="debug",
choices=["debug", "release"],
help="the settings preset for the import hook to use when building packages. Defaults to 'debug'.",
)
install.add_argument(
"--user",
action="store_true",
help="whether to install into usercustomize.py instead of sitecustomize.py. "
"Note that usercustomize.py is shared between virtualenvs of the same interpreter version and is not loaded "
"unless the virtualenv is created with the `--system-site-packages` argument. Use `site info` to check "
"whether usercustomize.py is loaded the current interpreter.",
)

uninstall = site_sub_actions.add_parser(
"uninstall",
help="uninstall the import hook from site-packages/sitecustomize.py or site-packages/usercustomize.py",
)
uninstall.add_argument(
"--user",
action="store_true",
help="whether to uninstall from usercustomize.py instead of sitecustomize.py",
)
site_sub_actions.add_parser("uninstall", help="uninstall the import hook from site-packages/sitecustomize.py")

args = parser.parse_args()

Expand All @@ -179,9 +224,9 @@ def _main() -> None:
if args.sub_action == "info":
_action_site_info(args.format)
elif args.sub_action == "install":
_action_site_install()
_action_site_install(user=args.user, preset_name=args.preset, force=args.force)
elif args.sub_action == "uninstall":
_action_site_uninstall()
_action_site_uninstall(user=args.user)
else:
site_action.print_help()
else:
Expand Down
Loading
Loading