Skip to content

Commit

Permalink
Depend on UV (#1393)
Browse files Browse the repository at this point in the history
  • Loading branch information
ofek authored Apr 16, 2024
1 parent c935e51 commit a229af7
Show file tree
Hide file tree
Showing 15 changed files with 164 additions and 77 deletions.
9 changes: 9 additions & 0 deletions docs/config/internal/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,12 @@ The `run` script is the default behavior while the `run-cov` script is used inst

!!! note
The `HATCH_TEST_ARGS` environment variable is how the [`test`](../../cli/reference.md#hatch-test) command's flags are translated and internally populated without affecting the user's arguments. This is also the way that [extra arguments](#extra-arguments) are passed.

### Installer

By default, [UV is enabled](../../how-to/environment/select-installer.md). You may disable that behavior as follows:

```toml config-example
[tool.hatch.envs.hatch-test]
uv = false
```
19 changes: 7 additions & 12 deletions docs/how-to/environment/select-installer.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,15 @@ The [virtual](../../plugins/environment/virtual.md) environment type by default
!!! warning "caveat"
UV is under active development and may not work for all dependencies.

To do so, enable the `uv` [option](../../plugins/environment/virtual.md#options). For example, if you wanted to enable this functionality for your [test environments](../../config/internal/testing.md#customize-environment), you could set the following:
To do so, enable the `uv` [option](../../plugins/environment/virtual.md#options). For example, if you wanted to enable this functionality for the [default](../../config/environment/overview.md#inheritance) environment, you could set the following:

```toml config-example
[tool.hatch.envs.hatch-test]
[tool.hatch.envs.default]
uv = true
```

The next time you use environments, Hatch will create a dedicated environment for UV (if it does not yet exist) which will be used for all subsequent environment creation and dependency resolution & installation.

!!! tip
All environments that enable UV will have `uv` available on PATH.
All environments that enable UV will have the path to `uv` available as the `HATCH_UV` environment variable.

## Configuring the version

Expand All @@ -34,7 +32,7 @@ dependencies = [

## Externally managed

If you want to manage UV yourself, you can expose it to Hatch by setting the `HATCH_ENV_TYPE_VIRTUAL_UV_PATH` environment variable. This should be the absolute path to a UV binary which Hatch will use instead of the internal environment. This implicitly [enables](#enabling-uv) the `uv` option.
If you want to manage UV yourself, you can expose it to Hatch by setting the `HATCH_ENV_TYPE_VIRTUAL_UV_PATH` environment variable which should be the absolute path to a UV binary for Hatch to use instead. This implicitly [enables](#enabling-uv) the `uv` option.

## Installer script alias

Expand All @@ -50,16 +48,13 @@ matrix.installer.uv = [
{ value = false, if = ["pip"] },
]
matrix.installer.scripts = [
{ key = "pip", value = "uv pip {args}", if = ["uv"] },
{ key = "pip", value = "{env:HATCH_UV} pip {args}", if = ["uv"] },
]
```

Another common use case is to enable UV for all [test environments](../../config/internal/testing.md). In this case, you often wouldn't want to modify the `scripts` mapping directly but rather add an [extra script](../../config/environment/overview.md#extra-scripts):
Another common use case is to expose UV to all [test environments](../../config/internal/testing.md). In this case, you often wouldn't want to modify the `scripts` mapping directly but rather add an [extra script](../../config/environment/overview.md#extra-scripts):

```toml config-example
[tool.hatch.envs.hatch-test]
uv = true

[tool.hatch.envs.hatch-test.extra-scripts]
pip = "uv pip {args}"
pip = "{env:HATCH_UV} pip {args}"
```
2 changes: 1 addition & 1 deletion docs/plugins/environment/virtual.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ type = "virtual"
| `python-sources` | `['external', 'internal']` | This may be set to an array of strings that are either the literal `internal` or `external`. External considers only Python executables that are already on `PATH`. Internal considers only [internally managed Python distributions](#internal-distributions). |
| `path` | | An explicit path to the virtual environment. The path may be absolute or relative to the project root. Any environments that [inherit](../../config/environment/overview.md#inheritance) this option will also use this path. The environment variable `HATCH_ENV_TYPE_VIRTUAL_PATH` may be used, which will take precedence. |
| `system-packages` | `false` | Whether or not to give the virtual environment access to the system `site-packages` directory |
| `uv` | `false` | Whether or not to use [UV](https://github.com/astral-sh/uv) in place of virtualenv & pip for virtual environment creation and dependency management, respectively. By default, Hatch will manage UV itself in an isolated environment. If you intend to provide UV instead, you may set the `HATCH_ENV_TYPE_VIRTUAL_UV_PATH` environment variable which should be the absolute path to a UV binary. This environment variable implicitly enables the `uv` option (if unset). |
| `uv` | `false` | Whether or not to use [UV](https://github.com/astral-sh/uv) in place of virtualenv & pip for virtual environment creation and dependency management, respectively. If you intend to provide UV yourself, you may set the `HATCH_ENV_TYPE_VIRTUAL_UV_PATH` environment variable which should be the absolute path to a UV binary. This environment variable implicitly enables the `uv` option (if unset). |

## Location

Expand Down
4 changes: 1 addition & 3 deletions hatch.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,10 @@ post-install-commands = [
]

[envs.hatch-test]
uv = true
extra-dependencies = [
"filelock",
"pyfakefs",
"trustme",
"uv",
# Hatchling dynamic dependency
"editables",
]
Expand All @@ -23,7 +21,7 @@ post-install-commands = [
extra-args = ["--dist", "worksteal"]

[envs.hatch-test.extra-scripts]
pip = "uv pip {args}"
pip = "{env:HATCH_UV} pip {args}"

[envs.coverage]
detached = true
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ dependencies = [
"tomli-w>=1.0",
"tomlkit>=0.11.1",
"userpath~=1.7",
"uv>=0.1.32",
"virtualenv>=20.16.2",
"zstandard<1",
]
Expand Down
31 changes: 4 additions & 27 deletions src/hatch/cli/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ def get_environment(self, env_name: str | None = None) -> EnvironmentInterface:
if environment_class is None:
self.abort(f'Environment `{env_name}` has unknown type: {environment_type}')

from hatch.env.internal import is_isolated_environment

if self.project.location.is_file():
data_directory = isolated_data_directory = self.data_dir / 'env' / environment_type / '.scripts'
elif is_isolated_environment(env_name, config):
Expand Down Expand Up @@ -344,6 +346,8 @@ def _write(self, environment: EnvironmentInterface, metadata: dict[str, Any]) ->
metadata_file.write_text(json.dumps(metadata))

def _metadata_file(self, environment: EnvironmentInterface) -> Path:
from hatch.env.internal import is_isolated_environment

if is_isolated_environment(environment.name, environment.config):
return self.__data_dir / '.internal' / f'{environment.name}.json'

Expand All @@ -352,30 +356,3 @@ def _metadata_file(self, environment: EnvironmentInterface) -> Path:
@cached_property
def _storage_dir(self) -> Path:
return self.__data_dir / self.__project_path.id


def is_isolated_environment(env_name: str, config: dict[str, Any]) -> bool:
# Provide super isolation and immunity to project-level environment removal only when the environment:
#
# 1. Does not require the project being installed
# 2. The default configuration is used
#
# For example, the environment for static analysis depends only on Ruff at a specific default
# version. This environment does not require the project and can be reused by every project to
# improve responsiveness. However, if the user for some reason chooses to override the dependencies
# to use a different version of Ruff, then the project would get its own environment.
if not config.get('skip-install', False):
return False

from hatch.env.internal import get_internal_env_config

internal_config = get_internal_env_config().get(env_name)
if not internal_config:
return False

# Only consider things that would modify the actual installation, other options like extra scripts don't matter
for key in ('dependencies', 'extra-dependencies', 'features'):
if config.get(key) != internal_config.get(key):
return False

return True
30 changes: 30 additions & 0 deletions src/hatch/env/internal/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,33 @@ def get_internal_env_config() -> dict[str, Any]:
internal_config[env_name] = env_config

return internal_config


def is_isolated_environment(env_name: str, config: dict[str, Any]) -> bool:
# Provide super isolation and immunity to project-level environment removal only when the environment:
#
# 1. Does not require the project being installed
# 2. The default configuration is used
#
# For example, the environment for static analysis depends only on Ruff at a specific default
# version. This environment does not require the project and can be reused by every project to
# improve responsiveness. However, if the user for some reason chooses to override the dependencies
# to use a different version of Ruff, then the project would get its own environment.
return config.get('skip-install', False) and is_default_environment(env_name, config)


def is_default_environment(env_name: str, config: dict[str, Any]) -> bool:
# Standalone environment
internal_config = get_internal_env_config().get(env_name)
if not internal_config:
# Environment generated from matrix
internal_config = get_internal_env_config().get(env_name.split('.')[0])
if not internal_config:
return False

# Only consider things that would modify the actual installation, other options like extra scripts don't matter
for key in ('dependencies', 'extra-dependencies', 'features'):
if config.get(key) != internal_config.get(key):
return False

return True
5 changes: 4 additions & 1 deletion src/hatch/env/internal/build.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
from __future__ import annotations

from typing import Any

def get_default_config() -> dict:

def get_default_config() -> dict[str, Any]:
return {
'skip-install': True,
'uv': True,
'dependencies': ['build[virtualenv]>=1.0.3'],
'scripts': {
'build-all': 'python -m build',
Expand Down
5 changes: 4 additions & 1 deletion src/hatch/env/internal/static_analysis.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
from __future__ import annotations

from typing import Any

def get_default_config() -> dict:

def get_default_config() -> dict[str, Any]:
return {
'skip-install': True,
'uv': True,
'dependencies': [f'ruff=={RUFF_DEFAULT_VERSION}'],
'scripts': {
'format-check': 'ruff format{env:HATCH_FMT_ARGS:} --check --diff {args:.}',
Expand Down
5 changes: 4 additions & 1 deletion src/hatch/env/internal/test.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
from __future__ import annotations

from typing import Any

def get_default_config() -> dict:

def get_default_config() -> dict[str, Any]:
return {
'uv': True,
'dependencies': [
'coverage-enable-subprocess==1.0',
'coverage[toml]~=7.4',
Expand Down
6 changes: 4 additions & 2 deletions src/hatch/env/internal/uv.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
from __future__ import annotations

from typing import Any

def get_default_config() -> dict:

def get_default_config() -> dict[str, Any]:
return {
'skip-install': True,
'dependencies': ['uv==0.1.31'],
'uv': True,
}
52 changes: 32 additions & 20 deletions src/hatch/env/virtual.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from hatch.env.utils import add_verbosity_flag
from hatch.utils.fs import Path
from hatch.utils.shells import ShellManager
from hatch.utils.structures import EnvVars
from hatch.venv.core import UVVirtualEnv, VirtualEnv

if TYPE_CHECKING:
Expand Down Expand Up @@ -80,36 +81,47 @@ def __init__(self, *args, **kwargs):

@cached_property
def use_uv(self) -> bool:
# Prevent recursive loop
if self.name == 'hatch-uv':
return False

return self.config.get('uv', bool(self.explicit_uv_path))

@cached_property
def explicit_uv_path(self) -> str:
return self.get_env_var_option('uv_path') or self.config.get('uv-path', '')

@cached_property
def virtual_env_cls(self) -> type[VirtualEnv]:
return UVVirtualEnv if self.use_uv else VirtualEnv

@cached_property
def uv_ready(self):
if not self.use_uv or self.explicit_uv_path:
def expose_uv(self):
if not (self.use_uv or self.uv_path):
return nullcontext()

uv_env = self.app.get_environment('hatch-uv')
self.app.prepare_environment(uv_env)
return uv_env
return EnvVars({'HATCH_UV': self.uv_path})

@cached_property
def uv_path(self) -> str:
if self.explicit_uv_path:
return self.explicit_uv_path

with self.uv_ready:
return self.platform.modules.shutil.which('uv')
from hatch.env.internal import is_default_environment

@cached_property
def explicit_uv_path(self) -> str:
return self.get_env_var_option('uv_path') or self.config.get('uv-path', '')
env_name = 'hatch-uv'
if not (
# Prevent recursive loop
self.name == env_name
# Only if dependencies have been set by the user
or is_default_environment(env_name, self.app.project.config.internal_envs[env_name])
):
uv_env = self.app.get_environment(env_name)
self.app.prepare_environment(uv_env)
with uv_env:
return self.platform.modules.shutil.which('uv')

import sysconfig

scripts_dir = sysconfig.get_path('scripts')
old_path = os.environ.get('PATH', os.defpath)
new_path = f'{scripts_dir}{os.pathsep}{old_path}'
return self.platform.modules.shutil.which('uv', path=new_path)

@staticmethod
def get_option_types() -> dict:
Expand Down Expand Up @@ -138,7 +150,7 @@ def create(self):
"""
)

with self.uv_ready:
with self.expose_uv():
self.virtual_env.create(self.parent_python, allow_system_packages=self.config.get('system-packages', False))

def remove(self):
Expand Down Expand Up @@ -186,7 +198,7 @@ def build_environment(self, dependencies):
from hatchling.dep.core import dependencies_in_sync

if not self.build_environment_exists():
with self.uv_ready:
with self.expose_uv():
self.build_virtual_env.create(self.parent_python)

with self.get_env_vars(), self.build_virtual_env:
Expand Down Expand Up @@ -226,7 +238,7 @@ def enter_shell(self, name: str, path: str, args: Iterable[str]):
with self.safe_activation():
self.platform.exit_with_command([path, *args])
else:
with self.get_env_vars():
with self.expose_uv(), self.get_env_vars():
shell_executor(path, args, self.virtual_env.executables_directory)

def check_compatibility(self):
Expand Down Expand Up @@ -436,7 +448,7 @@ def _python_constraint(self) -> SpecifierSet:
def safe_activation(self):
# In order of precedence:
# - This environment
# - UV environment (if enabled)
# - UV
# - User-defined environment variables
with self.get_env_vars(), self.uv_ready, self:
with self.get_env_vars(), self.expose_uv(), self:
yield
7 changes: 3 additions & 4 deletions src/hatch/venv/core.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import os
from tempfile import TemporaryDirectory

from hatch.env.utils import add_verbosity_flag, get_env_var_option
from hatch.env.utils import add_verbosity_flag
from hatch.utils.env import PythonInfo
from hatch.utils.fs import Path
from hatch.venv.utils import get_random_venv_name
Expand All @@ -26,7 +26,7 @@ def activate(self):
old_path = os.environ.pop('PATH', None)
self._env_vars_to_restore['PATH'] = old_path
if old_path is None:
os.environ['PATH'] = str(self.executables_directory)
os.environ['PATH'] = f'{self.executables_directory}{os.pathsep}{os.defpath}'
else:
os.environ['PATH'] = f'{self.executables_directory}{os.pathsep}{old_path}'

Expand Down Expand Up @@ -131,8 +131,7 @@ def __exit__(self, exc_type, exc_value, traceback):

class UVVirtualEnv(VirtualEnv):
def create(self, python, *, allow_system_packages=False):
uv_path = get_env_var_option(plugin_name='virtual', option='uv_path', default='uv')
command = [uv_path, 'venv', str(self.directory), '--python', python]
command = [os.environ.get('HATCH_UV', 'uv'), 'venv', str(self.directory), '--python', python]
if allow_system_packages:
command.append('--system-site-packages')

Expand Down
Loading

0 comments on commit a229af7

Please sign in to comment.