`uv add` hangs forever for complex pyproject

[alex-shapiro]

Summary

The uv add command is hanging forever for the following repo and commit: alex-shapiro/PufferLib@21807b3.

uv version: 0.9.10
OS version: macOS 15.6.1 (24G90)
command: uv add

Command log:

uv add --dev ruff -vvv
DEBUG uv 0.9.10 (Homebrew 2025-11-17)
TRACE Checking shared lock for `/Users/alex/.cache/uv` at `/Users/alex/.cache/uv/.lock`
DEBUG Acquired shared lock for `/Users/alex/.cache/uv`
DEBUG Found project root: `/Users/alex/Projects/PufferLib`
DEBUG No workspace root found, using project root
TRACE Checking lock for `/Users/alex/Projects/PufferLib` at `/var/folders/lz/jby64bjj69z_mwv96l9wz6c00000gn/T/uv-1247407c379164de.lock`
DEBUG Acquired lock for `/Users/alex/Projects/PufferLib`
DEBUG No Python version file found in workspace: /Users/alex/Projects/PufferLib
DEBUG Using Python request `==3.13.*` from `requires-python` metadata
DEBUG Checking for Python environment at: `.venv`
TRACE Found cached interpreter info for Python 3.13.0, skipping query of: .venv/bin/python3
DEBUG The project environment's Python version satisfies the request: `Python ==3.13.*`
TRACE The project environment's Python version meets the Python requirement: `==3.13.*`
TRACE The virtual environment's Python interpreter meets the Python preference: `prefer managed`
DEBUG Released lock at `/var/folders/lz/jby64bjj69z_mwv96l9wz6c00000gn/T/uv-1247407c379164de.lock`
TRACE Checking lock for `.venv` at `.venv/.lock`
DEBUG Acquired lock for `.venv`

Platform

macOS 15.6.1 (24G90)

Version

0.9.10 (Homebrew 2025-11-17)

Python version

Python 3.13.0

[zanieb]

It hangs at that log line?

Does using --no-sync make it complete immediately?

[alex-shapiro]

Yeah it hangs at exactly that log line. Adding --no-sync has no effect:

uv add --dev ruff --no-sync -vvv
DEBUG uv 0.9.10 (Homebrew 2025-11-17)
TRACE Checking shared lock for `/Users/alex/.cache/uv` at `/Users/alex/.cache/uv/.lock`
DEBUG Acquired shared lock for `/Users/alex/.cache/uv`
DEBUG Found project root: `/Users/alex/Projects/PufferLib`
DEBUG No workspace root found, using project root
DEBUG No Python version file found in workspace: /Users/alex/Projects/PufferLib
DEBUG Using Python request `==3.13.*` from `requires-python` metadata
DEBUG Checking for Python environment at: `.venv`
TRACE Found cached interpreter info for Python 3.13.0, skipping query of: .venv/bin/python3
DEBUG The project environment's Python version satisfies the request: `Python ==3.13.*`
TRACE The project environment's Python version meets the Python requirement: `==3.13.*`
TRACE The virtual environment's Python interpreter meets the Python preference: `prefer managed`
TRACE Checking lock for `.venv` at `.venv/.lock`
DEBUG Acquired lock for `.venv`

[zanieb]

This appears to be reproducible with just uv lock

❯ docker run -it astral/uv:bookworm bash -c "git clone https://github.com/alex-shapiro/PufferLib && cd PufferLib && git checkout 21807b34145822e307314dd0e3b503139c6aaa97 && uv lock -vv"
...
TRACE Querying interpreter executable at /root/.local/share/uv/python/cpython-3.13.9-linux-aarch64-gnu/bin/python3.13
DEBUG Released lock at `/root/.local/share/uv/python/.lock`
Using CPython 3.13.9

In which case the last log is emitted at

uv/crates/uv/src/commands/project/mod.rs

Lines 1009 to 1015 in 9a6eafc

	writeln!(
	printer.stderr(),
	"Using {} {}{}",
	implementation.pretty(),
	interpreter.python_version().cyan(),
	interpreter.variant().display_suffix().cyan(),
	)?;

which is here in the lock logic

uv/crates/uv/src/commands/project/lock.rs

Line 143 in 2d9fe7c

LockTarget::Workspace(workspace) => ProjectInterpreter::discover(

If using --frozen, it hangs at

DEBUG uv 0.9.10
TRACE Checking shared lock for `/root/.cache/uv` at `/root/.cache/uv/.lock`
DEBUG Acquired shared lock for `/root/.cache/uv`
DEBUG Found workspace root: `/PufferLib`
TRACE Discovering workspace members for: `/PufferLib`
DEBUG Adding root workspace member: `/PufferLib`

which is just above interpreter retrieval

uv/crates/uv/src/commands/project/lock.rs

Lines 131 to 134 in 2d9fe7c

	workspace =
	Workspace::discover(project_dir, &DiscoveryOptions::default(), &workspace_cache)
	.await?;
	LockTarget::Workspace(&workspace)

but the downstream logic for Frozen is quite simple?

uv/crates/uv/src/commands/project/lock.rs

Lines 331 to 351 in 2d9fe7c

	pub(super) async fn execute(self, target: LockTarget<'_>) -> Result<LockResult, ProjectError> {
	match self.mode {
	LockMode::Frozen => {
	// Read the existing lockfile, but don't attempt to lock the project.
	let existing = target
	.read()
	.await?
	.ok_or_else(|| ProjectError::MissingLockfile)?;
	// Check if the discovered workspace members match the locked workspace members.
	if let LockTarget::Workspace(workspace) = target {
	for package_name in workspace.packages().keys() {
	existing
	.find_by_name(package_name)
	.map_err(|_| ProjectError::LockWorkspaceMismatch(package_name.clone()))?
	.ok_or_else(|| {
	ProjectError::LockWorkspaceMismatch(package_name.clone())
	})?;
	}
	}
	Ok(LockResult::Unchanged(existing))
	}

[zanieb]

Oh, uhh this is suspicious? You have a massive conflict list

conflicts = [
    [
        { extra = "avalon" },
        { extra = "atari" },
        { extra = "box2d" },
        { extra = "bsuite" },
        { extra = "butterfly" },
        { extra = "classic_control" },
        { extra = "crafter" },
        { extra = "craftax" },
        { extra = "dm_control" },
        { extra = "dm_lab" },
        { extra = "griddly" },
        { extra = "kinetix" },
        { extra = "magent" },
        { extra = "microrts" },
        { extra = "minerl" },
        { extra = "minigrid" },
        { extra = "minihack" },
        { extra = "mujoco" },
        { extra = "nethack" },
        { extra = "nmmo" },
        { extra = "open_spiel" },
        { extra = "pokemon_red" },
        { extra = "procen" },
        { extra = "slimevolley" },
        { extra = "vizdoom" },
        { extra = "tribal-village" },
    ],
    [
    { extra = "mujoco" },
        { extra = "mujoco" },
        { extra = "cleanrl" }

    ]

https://github.com/alex-shapiro/PufferLib/blob/21807b34145822e307314dd0e3b503139c6aaa97/pyproject.toml#L264-L300

That creates a combinatorial explosion.

[zanieb]

I'm still a bit confused about where we're stuck though.

[zanieb]

If I delete the lockfile and resolve from scratch, we hang after resolution, e.g.:

root@33dda518e74d:/PufferLib# uv lock --dry-run    
Using CPython 3.13.9
Resolved 282 packages in 1.35s

Very peculiar.

[zanieb]

Adding logs to trace where we're at

We're stuck here

uv/crates/uv-resolver/src/lock/mod.rs

Lines 265 to 276 in ae1edef

	let mut package = Package::from_annotated_dist(dist, fork_markers, root)?;
	Self::remove_unreachable_wheels(resolution, &requires_python, node_index, &mut package);
	// Add all dependencies
	for edge in resolution.graph.edges(node_index) {
	let ResolutionGraphNode::Dist(dependency_dist) = &resolution.graph[edge.target()]
	else {
	continue;
	};
	let marker = *edge.weight();
	package.add_dependency(&requires_python, dependency_dist, marker, root)?;
	}

uv/crates/uv-resolver/src/lock/mod.rs

Lines 2453 to 2462 in ae1edef

	/// Add the [`AnnotatedDist`] as a dependency of the [`Package`].
	fn add_dependency(
	&mut self,
	requires_python: &RequiresPython,
	annotated_dist: &AnnotatedDist,
	marker: UniversalMarker,
	root: &Path,
	) -> Result<(), LockError> {
	let new_dep =
	Dependency::from_annotated_dist(requires_python, annotated_dist, marker, root)?;

This Dependency::from_annotated_dist does not return.

I believe it's this call in Dependency::new that's stuck

uv/crates/uv-resolver/src/lock/mod.rs

Line 4852 in ae1edef

complexified_marker: UniversalMarker::from_combined(complexified_marker),

It looks like we're calling the following function a lot of times

uv/crates/uv-pep508/src/marker/algebra.rs

Line 561 in 2fc5fe2

pub(crate) fn without_extras(&mut self, mut i: NodeId) -> NodeId {

It looks like we're just doing a ton of work, so I'd expect it to finish someday but something is clearly wrong here. Even if I edit some of these functions to be no-ops, the operation takes a very long time.

[zanieb]

I'd highly recommend trimming down the number of conflicts, are all of those packages actually mutually conflicting?

[alex-shapiro]

Yeah a lot of the extras have mutually incompatible dependency trees. Each extra is a reinforcement learning environment with a pinned, uniquely-outdated dependency on gym, gymnasium, pettingzoo, etc. The user only loads one environment at runtime (to train/eval a model on a specific environment), so the mutual exclusion block seemed like the best way to avoid dependency conflicts between multiple versions of those core package dependencies.

Do we know that the exclusion block is causing the hang? Dependency resolution when creating uv.lock takes 2.51 seconds, which seems reasonable.

[zanieb]

I think it's a problem with processing all the markers that are generated to represent the conflicts.