pipenv sync raises `ValueError: Path not found or generated` when `PIPENV_VENV_IN_PROJECT=1` and `.venv/bin` doesn't exist

[gabe-microsoft]

pipenv 2022.01.08 doesn't properly support creating a new virtual environment in the project (using PIPENV_VENV_IN_PROJECT=1) if there's isn't already an existing one. This makes it hard to do a clean install.

More specifically, if PIPENV_VENV_IN_PROJECT=1 and the .venv/bin directory doesn't exist within the project directory, pipenv sync raises ValueError: Path not found or generated rather than simply creating the directory (as was done in previous versions). See the repro steps below:

The initial state is a project directory with an initialized pipenv virtual environment

$ pipenv --version
pipenv, version 2022.1.8
$ echo $PIPENV_VENV_IN_PROJECT
1
$ pwd
/workspaces/project
$ pipenv --venv
/workspaces/project/.venv
$ pipenv --rm
$ pipenv sync --dev
Traceback (most recent call last):
  File "/home/vscode/.pyenv/versions/3.7.4/bin/pipenv", line 10, in <module>
    sys.exit(cli())
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/cli/command.py", line 657, in sync
    system=state.system
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/core.py", line 2842, in do_sync
    clear=clear,
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/core.py", line 531, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/core.py", line 452, in ensure_virtualenv
    python = ensure_python(project, three=three, python=python)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/core.py", line 344, in ensure_python
    path_to_python = find_a_system_python(python)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/core.py", line 313, in find_a_system_python
    python_entry = find_python(finder, line)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/utils.py", line 2230, in find_python
    result = finder.find_python_version(line)
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 288, in find_python_version
    return self.system_path.find_python_version(
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 120, in system_path
    self._system_path = self.create_system_path()
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 86, in create_system_path
    ignore_unsupported=self.ignore_unsupported,
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 683, in create
    instance = instance._run_setup()
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 251, in _run_setup
    paths[p] = new_instance.get_path(p.joinpath(bin_dir))
  File "/home/vscode/.pyenv/versions/3.7.4/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 439, in get_path
    raise ValueError("Path not found or generated: {0!r}".format(path))
ValueError: Path not found or generated: PosixPath('/workspaces/project/.venv/bin')
$ mkdir -p .venv/bin
$ pipenv sync --dev
Creating a virtualenv for this project...
# Output not shown, but pipenv successfully created the environment

As shown above:

• pipenv is configured to install the virtual environment within the project directory in the .venv dir. This is configured using PIPENV_VENV_IN_PROJECT=1
• We started with an existing pipenv virtual environment (i.e., the Pipfile.lock exists and the virtual environment is installed in .venv within the project directory)
• We run pipenv --rm to remove the existing environment (this removes the .venv dir)
• Running pipenv sync --dev now fails with ValueError: Path not found or generated
• After simply creating an empty directory .venv/bin, now pipenv sync --dev works fine

I do no know when this bug was introduced, but I know that it did not exist in 2018.11.26 but does exist in 2022.01.08.

[JCapriotti]

I just had a very similar issue, with the same version of Pipenv. , in my case it was complaining about a path that seemed to relate to a Pipenv environment I had just removed:

ValueError: Path not found or generated: PosixPath('/Users/jasonc/.local/share/virtualenvs/my-project-Nt-iynbk/bin')

May or may not be related, but I also just recently installed pyenv. I'm not familiar with how pyenv works internally, so I decided I'd try reinstalling Pipenv. I had originally installed it with pip install pipenv, this time I installed it with Homebrew.

That didn't work, but then I noticed that directory I mentioned above was in my PATH variable, so I closed/restarted my shell and the issue disappeared.

I have no idea how the PATH got messed up, that was my problem.

[gabe-microsoft]

@JCapriotti, this is a very interesting insight and it explains my issue as well!

I was using direnv which changes the PATH based on the virtual environment associated with the current directory. After I removed the virtual environment, the old virtual environment's path was still on the unchanged PATH variable. By running direnv reload or simply changing to a different directory and then back, direnv refreshed my PATH and then the error disappeared.

I will therefore close this issue, as it does not appear to be a pipenv bug.

[asmaier]

I believe this happens when you call pipenv --rm while you are still inside pipenv shell. This will mess up your PATH variable. So don't do this:

pipenv shell
pipenv --rm
pipenv install

Instead you must do this

pipenv shell
exit
pipenv --rm
pipenv install

[zhanglongqi]

Thank you. It works. Especially when I use byobu, I enabled vietualenv before start byobu, then I can not rebuild virtualenv inside byobu.

[roansong]

I ran into this after IntelliJ opened a terminal inside an activated virtualenv. Running set VIRTUAL_ENV "" (fish) or export VIRTUAL_ENV="" (bash) fixed it for me

[thernstig]

@gabe-microsoft I am in the same situation, but this was never a problem with previous version of pipenv.

To me, being able to do rm -rf .venv/ and then pipenv install --dev is something I want to work, but it gets messed up due to using direnv + PIPENV_VENV_IN_PROJECT=1.

I wonder if pipenv anyway should handle this.

If it does not find /workspaces/project/.venv/bin, it should just reinstall into that location. Pipenv did this previously. I think this should be re-opended.

[matteius]

Possibly a duplicate of: https://github.com/pypa/pipenv/issues/6019

[bkane11]

I also had to unset the VIRTUAL_ENV environment variable:

unset VIRTUAL_ENV

[Tlaloc-Es]

I have the same error with zsh in vscode, I only open the tilix from vscode, after that

rm -rf .venv
pipenv shell

Then closes the tilix and in the vscode terminal

deactivate
pipenv shell

and works again.