4.27.0 set_env value substitution from other section not honoring skip_missing_interpreters #3597
Description
Issue
Environments defined with a missing interpreter and whose set_env is substituting values from the root testenv raise an exception even if skip_missing_interpreter == True
ROOT: HandledError| replace failed in py312.set_env with RuntimeError("failed to find interpreter for Builtin discover of python_spec='py312'")
Environment
Running Python 3.10.12 from virtual env with Tox 4.27.0 installed on Ubuntu 22.04.5 LTS
Output of pip list of the host Python, where tox is installed
$pip3 -V
pip 24.0 from /home/steve/PycharmProjects/tox4-test/venv-3.10/lib/python3.10/site-packages/pip (python 3.10)
$pip3 list
Package Version
----------------------- -----------
annotated-types 0.7.0
astroid 3.3.11
backports.tarfile 1.2.0
betterproto 2.0.0b6
build 1.2.2.post1
cachetools 6.1.0
certifi 2024.2.2
cffi 1.16.0
chardet 5.2.0
charset-normalizer 3.3.2
colorama 0.4.6
cryptography 44.0.3
dill 0.4.0
distlib 0.3.8
dnspython 2.7.0
docutils 0.21.2
dumb-pypi 1.15.0
email_validator 2.2.0
exceptiongroup 1.2.1
filelock 3.18.0
git-filter-repo 2.47.0
grpclib 0.4.8
h2 4.2.0
hpack 4.1.0
hyperframe 6.1.0
id 1.5.0
idna 3.7
importlib_metadata 8.7.0
importlib-resources 5.13.0
iniconfig 2.0.0
isort 6.0.1
jaraco.classes 3.4.0
jaraco.context 5.3.0
jaraco.functools 4.0.1
jeepney 0.8.0
Jinja2 3.1.4
keyring 25.2.1
markdown-it-py 3.0.0
MarkupSafe 2.1.5
mccabe 0.7.0
mdurl 0.1.2
more-itertools 10.2.0
multidict 6.6.3
nh3 0.2.17
packaging 25.0
pbr 6.0.0
pip 24.0
pkg_about 1.3.6
pkginfo 1.10.0
platformdirs 4.3.8
pluggy 1.5.0
py 1.11.0
pyasn1 0.6.1
pycparser 2.22
pydantic 2.11.7
pydantic_core 2.33.2
Pygments 2.18.0
PyJWT 2.10.1
pylint 3.3.7
pyOpenSSL 25.1.0
pyproject-api 1.9.1
pyproject_hooks 1.2.0
pytest 8.2.1
pytest-pylint 0.21.0
python-dateutil 2.9.0.post0
PyYAML 6.0.1
readme_renderer 43.0
requests 2.32.3
requests-toolbelt 1.0.0
rfc3161-client 1.0.3
rfc3986 2.0.0
rfc8785 0.1.4
rich 13.7.1
SecretStorage 3.3.3
securesystemslib 1.3.0
setuptools 80.9.0
sigstore 3.6.4
sigstore-protobuf-specs 0.3.2
sigstore-rekor-types 0.0.18
six 1.17.0
tomli 2.2.1
tomlkit 0.13.3
tox 4.27.0
tox-backtick 0.6.5 <- my local copy, with changes not yet added via a PR.
tox-tags 0.2.0
tuf 6.0.0
twine 6.1.0
typing_extensions 4.14.0
typing-inspection 0.4.1
urllib3 2.2.1
virtualenv 20.31.2
wheel 0.43.0
zipp 3.23.0
Output of running tox
Output of tox -rvv
With python3.12 not found: (eg. which python3.12 -> null string ($? ==1)
$tox -rvv --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini list
py310: 263 I find interpreter for spec PythonSpec(major=3, minor=10, free_threaded=False) [virtualenv/discovery/builtin.py:76]
py310: 264 D discover exe for PythonInfo(spec=CPython3.10.12.final.0-64, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:451]
py310: 264 D filesystem is case-sensitive [virtualenv/info.py:27]
py310: 266 D got python info of /usr/bin/python3.10 from /home/steve/.local/share/virtualenv/py_info/2/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:132]
py310: 266 I proposed PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py310: 266 D accepted PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:85]
py312: 306 I find interpreter for spec PythonSpec(major=3, minor=12, free_threaded=False) [virtualenv/discovery/builtin.py:76]
py312: 306 I proposed PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 308 D discover PATH[0]=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin [virtualenv/discovery/builtin.py:152]
py312: 308 D discover PATH[1]=/opt/groovy-4.0.10/bin [virtualenv/discovery/builtin.py:152]
py312: 308 D discover PATH[2]=/usr/local/bin [virtualenv/discovery/builtin.py:152]
py312: 309 D discover PATH[3]=/usr/sbin [virtualenv/discovery/builtin.py:152]
py312: 311 D discover PATH[4]=/usr/bin [virtualenv/discovery/builtin.py:152]
py312: 315 D got python info of /usr/bin/python3 from /home/steve/.local/share/virtualenv/py_info/2/31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6.json [virtualenv/app_data/via_disk_folder.py:132]
py312: 316 I proposed PathPythonInfo(spec=CPython3.10.12.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 317 D discover PATH[5]=/sbin [virtualenv/discovery/builtin.py:152]
py312: 319 D discover PATH[6]=/bin [virtualenv/discovery/builtin.py:152]
py312: 323 D got python info of /bin/python3 from /home/steve/.local/share/virtualenv/py_info/2/916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1.json [virtualenv/app_data/via_disk_folder.py:132]
py312: 325 I proposed PathPythonInfo(spec=CPython3.10.12.final.0-64, exe=/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 326 D discover PATH[7]=/usr/games [virtualenv/discovery/builtin.py:152]
py312: 326 D discover PATH[8]=/snap/bin [virtualenv/discovery/builtin.py:152]
ROOT: 326 E HandledError| replace failed in py312.set_env with RuntimeError("failed to find interpreter for Builtin discover of python_spec='py312'") [tox/run.py:23]
Running the envlist so at least 1 testenv (py310) should work, py312 be ignored:
$tox -rvv --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini run
py310: 238 I find interpreter for spec PythonSpec(major=3, minor=10, free_threaded=False) [virtualenv/discovery/builtin.py:76]
py310: 239 D discover exe for PythonInfo(spec=CPython3.10.12.final.0-64, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:451]
py310: 239 D filesystem is case-sensitive [virtualenv/info.py:27]
py310: 241 D got python info of /usr/bin/python3.10 from /home/steve/.local/share/virtualenv/py_info/2/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:132]
py310: 241 I proposed PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py310: 241 D accepted PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:85]
py312: 276 I find interpreter for spec PythonSpec(major=3, minor=12, free_threaded=False) [virtualenv/discovery/builtin.py:76]
py312: 277 I proposed PythonInfo(spec=CPython3.10.12.final.0-64, system=/usr/bin/python3.10, exe=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 277 D discover PATH[0]=/home/steve/PycharmProjects/tox4-test/venv-3.10/bin [virtualenv/discovery/builtin.py:152]
py312: 278 D discover PATH[1]=/opt/groovy-4.0.10/bin [virtualenv/discovery/builtin.py:152]
py312: 278 D discover PATH[2]=/usr/local/bin [virtualenv/discovery/builtin.py:152]
py312: 279 D discover PATH[3]=/usr/sbin [virtualenv/discovery/builtin.py:152]
py312: 281 D discover PATH[4]=/usr/bin [virtualenv/discovery/builtin.py:152]
py312: 287 D got python info of /usr/bin/python3 from /home/steve/.local/share/virtualenv/py_info/2/31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6.json [virtualenv/app_data/via_disk_folder.py:132]
py312: 287 I proposed PathPythonInfo(spec=CPython3.10.12.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 289 D discover PATH[5]=/sbin [virtualenv/discovery/builtin.py:152]
py312: 292 D discover PATH[6]=/bin [virtualenv/discovery/builtin.py:152]
py312: 300 D got python info of /bin/python3 from /home/steve/.local/share/virtualenv/py_info/2/916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1.json [virtualenv/app_data/via_disk_folder.py:132]
py312: 301 I proposed PathPythonInfo(spec=CPython3.10.12.final.0-64, exe=/bin/python3, platform=linux, version='3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:83]
py312: 301 D discover PATH[7]=/usr/games [virtualenv/discovery/builtin.py:152]
py312: 302 D discover PATH[8]=/snap/bin [virtualenv/discovery/builtin.py:152]
ROOT: 303 E HandledError| replace failed in py312.set_env with RuntimeError("failed to find interpreter for Builtin discover of python_spec='py312'") [tox/run.py:23]
$echo $?
254If the substitution of {[testenv]set_env} commented out:
$tox --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini run
py310: commands_pre[0] GitRepos/ioif/lib/ioif/client-python> .tox/py310/lib/python3.10/site-packages/run-me
py310: Exception running subprocess [Errno 2] No such file or directory: '/home/steve/Documents/ART-002/.tox/py310/lib/python3.10/site-packages/run-me'
py310: exit 2 (0.00 seconds) /home/steve/Documents/ART-002> .tox/py310/lib/python3.10/site-packages/run-me
py310: command failed but is marked ignore outcome so handling it as success
py310: commands_pre[1] GitRepos/ioif/lib/ioif/client-python> bash -c -- 'echo envbindir is /home/steve/Documents/ART-002/.tox/py310/bin'
envbindir is /home/steve/Documents/ART-002/.tox/py310/bin
py310: commands[0] GitRepos/ioif/lib/ioif/client-python> python3 -V
Python 3.10.12
py310: commands[1] GitRepos/ioif/lib/ioif/client-python> bash -c -- 'echo running out of py310 with envtmpdir == /home/steve/Documents/ART-002/.tox/py310/tmp, envsitepackagesdir == /home/steve/Documents/ART-002/.tox/py310/lib/python3.10/site-packages'
running out of py310 with envtmpdir == /home/steve/Documents/ART-002/.tox/py310/tmp, envsitepackagesdir == /home/steve/Documents/ART-002/.tox/py310/lib/python3.10/site-packages
py310: OK (0.17=setup[0.07]+cmd[0.00,0.01,0.06,0.02] seconds)
congratulations :) (0.26 seconds)
(venv-3.10) steve~/Documents/ART-002/GitRepos/ioif/lib/ioif/client-python (tox4)*
$echo $?
0
offending substitution commented out:
$tox --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini list
default environments:
py310 -> [no description]
additional environments:
py312 -> [no description]
uncommented list:
$tox --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini list
ROOT: HandledError| replace failed in py312.set_env with RuntimeError("failed to find interpreter for Builtin discover of python_spec='py312'")
(venv-3.10) steve~/Documents/ART-002/GitRepos/ioif/lib/ioif/client-python (tox4)*
$echo $?
254
python3.12 bin location added to PATH:
$PATH=$PATH:/opt/py312/bin tox --override testenv.skip_missing_interpreter=True -c ~/Documents/ART-002/tox4-issue-tox.ini list
default environments:
py310 -> [no description]
additional environments:
py312 -> [no description]
(venv-3.10) steve~/Documents/ART-002/GitRepos/ioif/lib/ioif/client-python (tox4)*
$echo $?
0From my debugging session on Friday, stack frames when the replace failed exception is raised, sure looks like the nested replacement is not taking into account the skip_missing_interpreter setting. I wish I had a recommendation w/regard to a possible fix but it's early days for that, wanted to created this issue so perhaps those more familiar with the code might provided helpful feedback, ideas.
Minimal example
Here's a somewhat minimal tox.inv that will reproduce the issue: (used in the above output samples).
[tox]
envlist=py310
skip_missing_interpreters=true
[testenv]
allowlist_externals=
bash
set_env =
BASE=YES
# is is the need to replace stuff when referenced? Nope
VAR_NESTED={env:OUR_ontologies:{envdir}{/}ontologies}
# some inline JSON definition? Nope
TEST_API_KEYS=\{"ci-test": \{"uid": 1000\}\}
# what about backtick line that uses tox-backtick replacement? Nope
TWC_TIMEOUT=`+if [ "{env:DATA_SOURCE:TWC}" != "NONE" ] ; then echo {env:Job_TWC_TIMEOUT:30} ; else echo {env:Job_TWC_TIMEOUT:10} ; fi`
# any interpolation that needs virtual_env/api.py's VirtualEnv.env_site_package_dir method which returns the
# cast("Path", self.creator.purelib) <- self.creater is NOT a valid CPython object since it's not found.
DATA_DIR={envsitepackagesdir}{/}our_data{/}
TMP_DIR={envtmpdir}
commands_pre =
# apparently, only set_env replacement cause the issue.
-{envsitepackagesdir}/run-me
[testenv:{py310,py312}]
set_env =
#this next replacement usage commented, no problems.
{[testenv]set_env}
foo=yes
commands =
python3 -V
bash -c -- 'echo running out of {envname} with envtmpdir == {envtmpdir}, envsitepackagesdir == {envsitepackagesdir}'
commands_pre =
{[testenv]commands_pre}
bash -c -- 'echo envbindir is {envbindir}'