UnicodeDecodeError

[andreas-vester]

Current Behavior

I installed various Python versions with pyenv-win. I then set some local Python versions using pyenv local 3.11.4 3.10.11 3.9.13 3.8.10.

I installed nox via pipx.

Running nox gives me errors for all versions but 3.11.4.

$ nox -p 3.8
nox > Running session tests-3.8
nox > Creating virtual environment (virtualenv) using python3.8 in .nox\tests-3-8
nox > Command C:\Users\andre\.local\pipx\venvs\nox\Scripts\python.exe -m virtualenv 'C:\Users\andre\coding\df2img\.nox\tests-3-8' -p python3.8 failed with exit code 1:
Exception in thread Thread-2 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\subprocess.py", line 1597, in _readerthread
    buffer.append(fh.read())
                  ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 38: invalid start byte
Exception in thread Thread-4 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\andre\.pyenv\pyenv-win\versions\3.11.4\Lib\subprocess.py", line 1597, in _readerthread
    buffer.append(fh.read())
                  ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 38: invalid start byte
RuntimeError: failed to find interpreter for Builtin discover of python_spec='python3.8'
nox > Session tests-3.8 failed.

Here's my noxfile.py:

"""Nox sessions."""
import os

import nox
from nox.sessions import Session

os.environ.update({"PDM_IGNORE_SAVED_PYTHON": "1"})
nox.options.sessions = [
    "pre-commit",
    "tests",
]
PYTHON_VERSIONS = ["3.8", "3.9", "3.10", "3.11"]


@nox.session(name="pre-commit", python="3.11")
def precommit(session: Session) -> None:
    """Lint using pre-commit."""
    args = session.posargs or [
        "run",
        "--all-files",
        "--hook-stage=manual",
    ]
    session.run("pdm", "install", "-G", "lint", external=True)
    session.run("pre-commit", "install")
    session.run("pre-commit", *args)


@nox.session(name="tests", python=PYTHON_VERSIONS)
def tests(session: Session) -> None:
    """Run the test suite."""
    args = session.posargs or ["--cov"]

    # install the package itself into a new virtual environment with tests dependencies
    session.run("pdm", "install", "-G", "test", external=True)
    # run pytest against the installed package
    session.run("pytest", *args)

Expected Behavior

I expect unit tests to run for all Python versions.

Steps To Reproduce

No response

Environment

- OS: Win 11
- Python: 3.11.4, 3.10.11, 3.9.13, 3.8.10
- Nox: 2023.4.22

Anything else?

No response

[henryiii]

I don't think we support pyenv's shims very well (I think most tools don't, at least last I checked). I expect nox is installed with 3.11 and so it's working with the 3.11 one, and the others are not being resolved correctly from the shims.

[cjolowicz]

Can you reproduce the UTF-8 decode error with virtualenv alone?

C:\Users\andre\.local\pipx\venvs\nox\Scripts\python.exe -m virtualenv venv -p python3.8

This looks like an upstream bug in either virtualenv or pyenv-win.

[andreas-vester]

Can you reproduce the UTF-8 decode error with virtualenv alone?

C:\Users\andre\.local\pipx\venvs\nox\Scripts\python.exe -m virtualenv venv -p python3.8

This looks like an upstream bug in either virtualenv or pyenv-win.

Yes, I can reproduce this error with virtualenv alone.