Interpreter discovery bug wrt. Microsoft Store shortcut #2812
Comments
|
I came here from pyenv-win. Mind trying a different patch? |
Seems to work just as well. Maybe Bernát can make a call on how he would like to have this handled, once he's back. |
|
Hi @axel-kah, Here's a suggested solution for this issue: # in virtualenv/discovery/cached_py_info.py
import sys
import subprocess
from subprocess import PIPE
... other imports
def from_exe(exe, env=None, raise_on_error=True):
"""
Given a python executable, get the python information as a dictionary
"""
cmd = [exe, "-c", PY_INFO_CODE]
env = env or {}
# NEW: Set encoding to cp1252 on Windows
encoding = "cp1252" if sys.platform == "win32" else "utf-8"
try:
process = subprocess.Popen(cmd, stdout=PIPE, stderr=PIPE, env=env) # MODIFIED
except OSError as os_error:
if raise_on_error:
raise
return {"error": os_error}
out, err = process.communicate()
try: # MODIFIED
out = out.decode(encoding)
err = err.decode(encoding)
except UnicodeDecodeError: # catch the exception and return meaningful error information
return {"error": f"Failed to decode output using {encoding} encoding: {out!r}, {err!r}"}
# ... rest of the function (unchanged)Explanation: The original code in The solution changes the decoding to use CP1252 on Windows. The modified line specifically sets the encoding = "cp1252" if sys.platform == "win32" else "utf-8"Then, the output of the subprocess is decoded using this dynamically determined encoding: try:
out = out.decode(encoding)
err = err.decode(encoding)
except UnicodeDecodeError: # Handles potential issues even with cp1252
return {"error": f"Failed to decode output using {encoding} encoding: {out!r}, {err!r}"}This allows the code to correctly handle the output from the Microsoft Store Python stub, even if it contains characters not representable in UTF-8. The This fix targets the root cause of the issue within |
Is that true in all locales, or does it actually use the appropriate locale encoding (which would be more in line with general Windows practice)? Would it be better here to use the locale encoding? On the other hand, if we don’t care about non-ASCII bytes in the output, and all that matters is to avoid decode errors, any encoding for which all bytes are valid would work. In that case, CP1252 should be fine, although Latin-1 is conventionally used because it maps all bytes to the same code point. |
Issue
☹️
hatchis usingvirtualenvs interpreter discovery during creation of its virtual envs. The discovery also finds the Microsoft Store python shortcut. Even though the interpreter was not installed using the MS Store, this executable is used during discovery to run virtualenvspy_info.pyscript. In this setting,hatchis able to successfully create its venv (read: exit code 0), but the discovery returns a bunch ofUnicodeDecodeErrorsand spills them on the terminalThe root cause seems to be that
virtualenvis spawning a subprocess for each interpreter it finds and has it execute thepy_info.pyscript. On Windows this will also try the same with the "mysterious"C:\Users\axel-kah\AppData\Local\Microsoft\WindowsApps\python.exe. If python was not installed using MS Store, then this executable will return an error message using the infamouscp1252encoding. When the OS is set to using a language like german, then this error message will contain german umlauts likeüwhich result in theUnicodeDecodeErrors.Proposed Fix
Change the encoding to
cp1252when on windows when launching the subprocesses during discovery, instead of usingutf-8for all platforms.I have verified the fix by locally patching a dev install of
hatchand could submit a PR.Environment
Provide at least:
hatch1.13.0virtualenv20.28.0pip listof the host python wherevirtualenvis installed:Output of the virtual environment creation
Not applicable because venv is created implicitly by hatch.
The text was updated successfully, but these errors were encountered: