Is it possible to detect and preserve packages from base virtual environment in multi-stage Docker builds?

[agusticonesagago]

Question: Is it possible for Poetry to validate existing virtual environment packages?

Context

I'm using Docker multi-stage builds where a base image contains a Poetry-managed virtual environment, and derived images need to add dependencies while reusing the base environment.

Current Behavior

When poetry install runs in a derived container:

• It installs packages based on the new pyproject.toml
• It doesn't detect or validate existing packages in the virtual environment
• Installation completes successfully even when there are version incompatibilities
• pip check afterwards reveals actual dependency conflicts that Poetry didn't catch

Specific Example

Base Image: Virtual environment with Package A == 2.0
Derived Image: pyproject.toml specifies Package A == 1.5
Result: Poetry installs successfully but pip check shows conflicts

Core Questions

1. Is it currently possible for Poetry to:

• Detect existing packages in a virtual environment before installation?
• Validate compatibility between existing and required package versions?
• Report conflicts during the installation process instead of silently succeeding?

2. Is this scenario supported by Poetry's current design, or is it considered outside the intended use case?

3. Are there built-in Poetry commands or flags that can perform this validation?

4. Should Poetry be expected to catch these incompatibilities that pip check detects?

Why This Matters

This pattern is increasingly common in:

• Docker multi-stage builds with shared dependency layers
• Development containers extending production base images
• CI/CD pipelines with virtual environment caching

Environment

• Poetry version: 2.1.2
• Python version: 3.11
• Platform: Docker containers (Debian GNU/Linux 11)

Bottom line: Is Poetry capable of this validation, or should I be using different tools/approaches for this use case?

[dimbleby]

poetry only knows or cares about packages in its lockfile

if you are claiming that locked packages are incompatible, that would be a bug (but in that case you likely are mistaken about what is happening)

if you mean that poetry will happily install things that clash with something about which it knows nothing - that's correct. It's up to you to manage those other things. If you want poetry to care about them, say so in your pyproject.toml.

[agusticonesagago]

Is there any way to reuse the same virtual environment if the base image already has a Poetry venv, and make Poetry manage what was already installed there — even if you have two different lock files?

[radoering]

I would say your second lock file should include everything that your first lock file contains (+ additional stuff). Then, you can use poetry sync. Detecting conflicts between two independent lock files is out of scope for Poetry.

[agusticonesagago]

I wanted to use a base image across multiple projects, keeping the pyproject.toml in each project very small and only installing the project-specific dependencies on top. That seems to mean Poetry isn’t really designed with base images in mind, right?

[radoering]

From Poetry's point of view: Your project should only require project-specific dependencies. It does not need any other dependencies so why should it care about them? If your project requires dependencies that are not in the pyproject.toml that is considered a bug in the pyproject.toml (user issue).

[radoering]

When poetry install runs in a derived container:

* It installs packages based on the new `pyproject.toml`
* It doesn't detect or validate existing packages in the virtual environment
* Installation completes successfully even when there are version incompatibilities

poetry install does not care about existing packages that are not in the lockfile. It just makes sure everything in the lockfile is installed. (If there is not a bug, there cannot be conflicts between packages that are in the lockfile.) If you want to uninstall packages that are not in the lockfile, you can use poetry sync. If you want to detect conflicts between locked packages and existing packages in the venv that are not part of the lockfile, then that is out of scope for Poetry.

[agusticonesagago]

So what you’re saying is that the only way is to manually check which dependencies the base image already has (at the Poetry level) and then put those into the project’s pyproject.toml? And there’s no plan for Poetry to be able to do this automatically by detecting what’s already in the venv?

I imagine there must be similar use cases from people relying on base images. Is anything like this on the Poetry roadmap?

[radoering]

Similar (but maybe different) use cases may be:

• FAQ: Poetry busts my Docker cache because it requires me to COPY my source files in before installing 3rd party dependencies
• using virtualenvs.options.system-site-packages to use large pre-installed packages