Display download status instead of "Locking..." during package install

[DaWe35]

Today I was trolled by this "Locking..." message and I find that it's actually a couple hundred developers who don't understand why CPU usage is 0 during install: https://stackoverflow.com/questions/56440090/pipenv-stuck-locking

As it turned out, Pipenv was actually downloading a big file. I think it would be better to display a download status when downloading a big file, so developers won't think pipenv is doing nothing.

Reproduce:

pipenv shell
pipenv install torch

[matteius]

@DaWe35 what version of pipenv are you running? Technically by calling pipenv install you are invoking a Locking phase initially. If you wanted to install directly from the lock file that would be pipenv sync

I see different output than what you are describing anyway on the latest pipenv:

$ pipenv install torch
Installing torch...
Resolving torch...
Adding torch to Pipfile's [packages] ...
Installation Succeeded
Installing dependencies from Pipfile.lock (d3ad1c)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

[DaWe35]

I think it should be the latest release, but I can't check since I already removed it. I was very upset about pipenv not using the python version configured with pyenv.

[matteius]

I was very upset about pipenv not using the python version configured with pyenv.

Ok well that is another indicator you probably weren't on the latest release. I released pythonfinder improvements in that space.

matteius@matteius-VirtualBox:~/pipenv-triage/issue-5718$ pyenv version
3.11.2 (set by /home/matteius/.python-version)
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5718$ pipenv install requests
Creating a virtualenv for this project...
Pipfile: /home/matteius/pipenv-triage/issue-5718/Pipfile
Using /home/matteius/.pyenv/versions/3.11.2/bin/python3 (3.11.2) to create virtualenv...
⠹ Creating virtual environment...created virtual environment CPython3.11.2.final.0-64 in 160ms
  creator CPython3Posix(dest=/home/matteius/.virtualenvs/issue-5718-IIGvo9D9, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/matteius/.local/share/virtualenv)
    added seed packages: pip==23.1.2, setuptools==67.7.2, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/matteius/.virtualenvs/issue-5718-IIGvo9D9
Installing requests...
Resolving requests...
Adding requests to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (665e1134856b92e638eb7ba89776f1a8b38ecd3d0ffd76d3964567e89484f330)!
Installing dependencies from Pipfile.lock (84f330)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Its possible that based on your PATH preferences it would still find your system python first without any other specifiers, but for example if your system python is 3.10 and you want 3.11 from pyenv, you should pass --python 3.11 when issuing the first pipenv command to create the virtualenv.

[DaWe35]

Okay so I think the problem was I installed pipenv and then pyenv, so pipenv was already there when I installed the 3.10 python. (My global system had 3.11). Does it mean it's impossible to change python version after the first pipenv command was issued?

[matteius]

It does not matter the order of pipenv and pyenv install, but it is true that it will often prefer system python over pyenv or asdf -- unless you specify otherwise.

Once the virtualenv is created for a project space with a version of python you do not prefer, you have to first remove it with pipenv --rm and then re-create it. Then check that the generated Pipfile doesn't call out the wrong version of python that you want. If it does you can either remove this section or you could adjust the specifier there to be what you want so that you don't have to specify on the CLI the --python flag.

Pipenv uses pythonfinder to determine which python to use, and by default it will just find the first default one on the PATH I believe, which is typically the system python. However with pyenv or asdf installed and specifying a version you don't have installed should prompt it to be installed, and this will work normally in one command.

However for pre-release python versions, it does work currently but requires issuing two commands. For example, if I want to try latest 3.12 and have pyenv installed:

$ pipenv install requests --python 3.12.0b1
Warning: Python 3.12.0b1 was not found on your system...
Would you like us to install CPython 3.12.0b1 with Pyenv? [Y/n]: Y
Installing CPython 3.12.0b1 with C:\Users\matte\.pyenv\pyenv-win\bin\pyenv.BAT (this may take a few minutes)...
Success!
[====] Installing python...:: [Info] ::  Mirror: https://www.python.org/ftp/python
:: [Installing] ::  3.12.0b1 ...
:: [Info] :: completed! 3.12.0b1

There is an edge case right now unfortunately on the pre-release version numbers like my above example 3.12.0b1 because I had to specify 3.12.0b1 to get pyenv to install the pre-release, but then pythonfinder needs just 3.12 or 3.12.0 to find it:

Warning: The Python you just installed is not available on your PATH, apparently.

So re-issuing the command with --python 3.12.0 allows working around this pre-release specifier issue:

matte@LAPTOP-N5VSGIBD MINGW64 ~/Projects/pipenv-triage/issue-5718
$ pipenv install requests --python 3.12.0
Creating a virtualenv for this project...
Pipfile: C:\Users\matte\Projects\pipenv-triage\issue-5718\Pipfile
Using c:/users/matte/.pyenv/pyenv-win/versions/3.12.0b1/python3.exe (3.12.0) to create virtualenv...
[    ] Creating virtual environment...created virtual environment CPython3.12.0.beta.1-64 in 2858ms
  creator CPython3Windows(dest=C:\Users\matte\.virtualenvs\issue-5718-v26YOnrf, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, via=copy, app_data_dir=C:\Users\matte\AppData\Local\pypa\virtualenv)
    added seed packages: pip==23.1.2
  activators BashActivator,BatchActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

Successfully created virtual environment!
Virtualenv location: C:\Users\matte\.virtualenvs\issue-5718-v26YOnrf
Creating a Pipfile for this project...
Installing requests...
Resolving requests...
Adding requests to Pipfile's [packages] ...
Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (238262fc123a448f5636a8e03c5a139138a6bb719558e6a3867b751ddfee5a6b)!
Installing dependencies from Pipfile.lock (ee5a6b)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

I hope that helps.

I closed this as not planned currently, for a few reasons that we cannot display the download status currently: 1.) downloads would happen in requirementslib and pip -- requirementslib is under major overhaul right now. 2.) we use multi-processing in some places making it harder to display a progress -- the old pipenv had a progress bar and it was impacting overall performance of pipenv and so it was removed not only for performance, but because its progress status was very misleading as well.

Maybe one day we can re-consider ways to add progress indicators, but it is not top of mind for the project right now and we generally advice if you want more progress indicator to run with the --verbose flag to see more output.

[dandv]

Can this issue be reopened as a feature request please?

Thousands of developers are confused by this "Locking..." status, when pipenv could display a progress of some sort if downloading anything has been taking more than 10 seconds.

Even with --verbose, I have no idea how long downloading a package will take ("go make some coffee" vs. "go for a run" type of long). The developer experience could definitely be improved here.

[tomskyPotato]

For that reason i left pipenv and went back to venv :-(