Main icon indicating copy to clipboard operation
Main copied to clipboard

[Bug] Pip upgrade failed

Open gianluca-sabena opened this issue 2 years ago • 17 comments

Bug Report

  • Install python with scoop install python
  • Python state pip need an upgrade and suggest to run
[notice] A new release of pip is available: 23.1.2 -> 23.2.1
[notice] To update, run: python.exe -m pip install --upgrade pip 
  • I run python.exe -m pip install --upgrade pip

Current Behavior

Pip does not upgrade and fails with an error

 python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\....\scoop\apps\python\current\lib\site-packages (23.1.2)
Collecting pip
  Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.1.2
    Uninstalling pip-23.1.2:
      Successfully uninstalled pip-23.1.2
  WARNING: Failed to write executable - trying to use .deleteme logic
  Rolling back uninstall of pip
  Moving to c:\users\....\scoop\persist\python\lib\site-packages\pip-23.1.2.dist-info\
   from C:\Users\....\scoop\persist\python\Lib\site-packages\~ip-23.1.2.dist-info
  Moving to c:\users\....\scoop\persist\python\lib\site-packages\pip\
   from C:\Users\....\scoop\persist\python\Lib\site-packages\~ip
  Moving to c:\users\...\scoop\persist\python\scripts\
   from C:\Users\...\scoop\persist\python\~cripts
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\....\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\....\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'

Expected Behavior

I expect pip to upgrade

System details

Windows version: 10

OS architecture: 64bit

PowerShell version: 7.3.6

Scoop Configuration

{
  "last_update": "2023-08-11T15:04:01.8110616+02:00",
  "scoop_repo": "https://github.com/ScoopInstaller/Scoop",
  "scoop_branch": "master"
}

gianluca-sabena avatar Aug 11 '23 14:08 gianluca-sabena

I have the same exact issuse except during the installation itself.

scoop install python  # (current version is 3.11.5)

...

Running post_install script...
  WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Zambo\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\Zambo\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'

...

'python' (3.11.5) was installed successfully!

Oddly enough, this does NOT happen if running:

scoop install versions/python311

OS specs:

image

wtfzambo avatar Aug 28 '23 14:08 wtfzambo

@gianluca-sabena could you try running pip install --upgrade pip or pip install --upgrade pip --user?

rashil2000 avatar Sep 10 '23 20:09 rashil2000

I have upgraded python last week with scoop, I have the latest version of pip now, the test is not significant.

> pip install --upgrade pip
Requirement already satisfied: pip in c:\users\......\scoop\apps\python\current\lib\site-packages (23.2.1)

gianluca-sabena avatar Sep 11 '23 09:09 gianluca-sabena

So this is no longer an issue?

rashil2000 avatar Sep 12 '23 10:09 rashil2000

I don't know... To recaps:

  • A) if you upgrade python, pip is upgraded (I suppose it ships directly with python) -> this has never been an issue at least for me (I read another user had a different problem https://github.com/ScoopInstaller/Main/issues/5090#issuecomment-1712925456)
  • B) if there is a new release of pip and you try to upgrade only pip with python.exe -m pip install --upgrade pip -> this was the issue I had

Since I upgraded python (case A) I have the latest pip version an I can't test the (case B) right now. I have to wait a new release of pip. I put python on hold so I will test case B in the future. This will probably take two months.

A possible approach to reproduce the bug could be:

  • install an old version of python 3.9.x (one that doesn't include the latest version of pip)
  • try to upgrade only pip with pip install...

gianluca-sabena avatar Sep 12 '23 16:09 gianluca-sabena

I see.

I'll reopen this issue in case (B) happens again.

rashil2000 avatar Sep 28 '23 15:09 rashil2000

I am currently getting the same issue while trying to install python

https://github.com/ScoopInstaller/Main/issues/5090#issuecomment-1712925456

Running post_install script...
  WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Justin\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\Justin\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'

StandBackBurrito avatar Oct 05 '23 00:10 StandBackBurrito

@gianluca-sabena could you try running pip install --upgrade pip or pip install --upgrade pip --user?

pip install --upgrade pip or pip install --upgrade pip --user did not work for me but this did:

(make sure to replace the username)

C:\Users\D\scoop\apps\python\current\python.exe -m pip install --upgrade pip --user:

Here are the more universal(?) commands:

pwsh: ~\scoop\apps\python\current\python.exe -m pip install --upgrade pip --user

cmd: %userprofile%\scoop\apps\python\current\python.exe -m pip install --upgrade pip --user

baula-party avatar Oct 18 '23 01:10 baula-party

I helded python packages to python 3.11.5 main 2023-09-11 11:35:42 Held package and I upgrade pip successfully with this command python.exe -m pip install --upgrade pip

python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\......\scoop\apps\python\current\lib\site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/e0/63/b428aaca15fcd98c39b07ca7149e24bc14205ad0f1c80ba2b01835aedde1/pip-23.3-py3-none-any.whl.metadata
  Downloading pip-23.3-py3-none-any.whl.metadata (3.5 kB)
Downloading pip-23.3-py3-none-any.whl (2.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 2.0 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-23.3

I confirm the issue is now resolved

gianluca-sabena avatar Oct 20 '23 06:10 gianluca-sabena

This is still happening. Yes, 'C:\Users\Ben\scoop\apps\python\current\python.exe' -m pip install --upgrade pip --user will work - I guess - but when using scoop I shouldn't have to pass --user

Ben@t480-ben ~$ python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\ben\scoop\apps\python\current\lib\site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/15/aa/3f4c7bcee2057a76562a5b33ecbd199be08cdb4443a02e26bd2c3cf6fc39/pip-23.3.2-py3-none-any.whl.metadata
  Using cached pip-23.3.2-py3-none-any.whl.metadata (3.5 kB)
Using cached pip-23.3.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
  WARNING: Failed to write executable - trying to use .deleteme logic
  Rolling back uninstall of pip
  Moving to c:\users\ben\scoop\persist\python\lib\site-packages\pip-23.2.1.dist-info\
   from C:\Users\Ben\scoop\persist\python\Lib\site-packages\~ip-23.2.1.dist-info
  Moving to c:\users\ben\scoop\persist\python\lib\site-packages\pip\
   from C:\Users\Ben\scoop\persist\python\Lib\site-packages\~ip
  Moving to c:\users\ben\scoop\persist\python\scripts\
   from C:\Users\Ben\scoop\persist\python\~cripts
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Ben\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\Ben\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'


[notice] A new release of pip is available: 23.2.1 -> 23.3.2
[notice] To update, run: python.exe -m pip install --upgrade pip
Ben@t480-ben ~$ python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\ben\scoop\apps\python\current\lib\site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/15/aa/3f4c7bcee2057a76562a5b33ecbd199be08cdb4443a02e26bd2c3cf6fc39/pip-23.3.2-py3-none-any.whl.metadata
  Using cached pip-23.3.2-py3-none-any.whl.metadata (3.5 kB)
Using cached pip-23.3.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
  WARNING: Failed to write executable - trying to use .deleteme logic
  Rolling back uninstall of pip
  Moving to c:\users\ben\scoop\persist\python\lib\site-packages\pip-23.2.1.dist-info\
   from C:\Users\Ben\scoop\persist\python\Lib\site-packages\~ip-23.2.1.dist-info
  Moving to c:\users\ben\scoop\persist\python\lib\site-packages\pip\
   from C:\Users\Ben\scoop\persist\python\Lib\site-packages\~ip
  Moving to c:\users\ben\scoop\persist\python\scripts\
   from C:\Users\Ben\scoop\persist\python\~cripts
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Ben\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\Ben\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'


[notice] A new release of pip is available: 23.2.1 -> 23.3.2
[notice] To update, run: python.exe -m pip install --upgrade pip
Ben@t480-ben ~$ 'C:\Users\Ben\scoop\apps\python\current\python.exe' -m pip install --upgrade pip --user
Requirement already satisfied: pip in c:\users\ben\scoop\apps\python\current\lib\site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/15/aa/3f4c7bcee2057a76562a5b33ecbd199be08cdb4443a02e26bd2c3cf6fc39/pip-23.3.2-py3-none-any.whl.metadata
  Using cached pip-23.3.2-py3-none-any.whl.metadata (3.5 kB)
Using cached pip-23.3.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  WARNING: The scripts pip.exe, pip3.12.exe and pip3.exe are installed in 'C:\Users\Ben\AppData\Roaming\Python\Python312\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.3.2

jcrben avatar Jan 31 '24 10:01 jcrben

I hit this and I debugged the problem.

A quick workaround

Create an empty file called placeholder.txt in %USERPROFILE%\scoop\apps\python\current\scripts\. (Name and contents don't matter, it only matters that the directory contains something other than pip's own scripts.)

After that, python -m pip install --upgrade pip will work just fine.

What is causing the problem

Upgrading pip means uninstalling the old version and installing the new version.

To uninstall a package, pip tries to delete all the package's files. But since later actions can fail, pip doesn't delete them immediately -- instead it moves the files to be deleted to temporary directories, then either deletes those (commit) or renames them back (rollback).

When you uninstall pip, three of the files to be uninstalled are (e.g. for Python 3.12):

scoop\apps\python\current\Scripts\pip.exe
scoop\apps\python\current\Scripts\pip3.exe
scoop\apps\python\current\Scripts\pip3.12.exe

Elsewhere in pip there's an optimization where, if you are deleting all the contents of a directory, it will try to stage the directory for delete/rollback instead of the individual files. See here in compress_for_rename.

If the only files in Scripts are pip's own -- which is not uncommon -- then trying to uninstall pip will try to rename the Scripts directory away. This would work as expected if Scripts were a plain directory, but thanks to some cleverness on Scoop's part it's not -- it's a junction point (the Windows version of a symbolic link):

C:\Users\mmdriley>dir C:\Users\mmdriley\scoop\apps\python\current\*Scripts*
...
05/17/2024  08:07 PM    <JUNCTION>     Scripts [\??\C:\Users\mmdriley\scoop\persist\python\Scripts]

Some other code in pip proactively normalizes paths (which resolves symbolic links) when adding files to be deleted, so the set of files that goes into compress_for_rename already points into scoop\persist\python\Scripts\. The net effect is pip renames scoop\persist\python\Scripts to scoop\persist\python\~cripts but the scoop\apps\python\current\Scripts symlink is left unchanged -- which means it's left pointing to a now-nonexistent directory.

Later, the install part of the upgrade comes along and tries to write the new pip scripts into scoop\apps\python\current\Scripts\ -- see the call to write_binary_file here. The write_binary_file function, in turn, diligently tries to ensure the containing directory exists before writing the file. Here, that check succeeds because the Scripts symlink exists -- but then when pip tries to write a file into the Scripts directory, the symlink fails to resolve, and boom!

Why this fixes it

Adding another file to the Scripts directory stops compress_for_rename from deciding that pip should delete the entire Scripts directory, avoids corrupting the symlink, and allows pip to finish the install successfully.

Recommendation for scoop

Maybe the easiest way to keep the junction point "persist" mechanism and avoid this edge case is just to add a placeholder.txt file in Scripts as part of installing the Python Scoop package?

mmdriley avatar May 18 '24 06:05 mmdriley

Thank you @mmdriley !!! At time of writing any clean Scoop installation of Python 3.12 currently ends with this issue if you try to upgrade pip right after. Kudos for this complete investigation 🍻

Note: Now I know the root cause, my workaround was to install another script (Black, Typer, Ruff, etc.) before upgrading pip itself.

Mohorelien avatar Jul 01 '24 20:07 Mohorelien

As the official main bucket hasn't adoped the fix yet (and the error still happens when updating Python), I think that we should reopen the issue and maybe start a PR in the main bucket adding the fix.

rjys365 avatar Apr 28 '25 07:04 rjys365

i've just had this happen. why has this issue been closed unresolved?

BoringBoredom avatar Jun 04 '25 04:06 BoringBoredom

@HUMORCE fyi

jcrben avatar Jun 10 '25 01:06 jcrben

just use

pip3 install --upgrade pip --user

pip --version

fcdbx23 avatar Nov 14 '25 12:11 fcdbx23

I just installed Scoop yesterday, 24-Nov-2025, and then installed Python 3.14.0. Pip has an update and it fails in the way reported above.

C:\Users\Han> C:\Users\Han\scoop\apps\python\current\python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\han\scoop\apps\python\current\lib\site-packages (25.2)
Collecting pip
  Using cached pip-25.3-py3-none-any.whl.metadata (4.7 kB)
Using cached pip-25.3-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 25.2
    Uninstalling pip-25.2:
      Successfully uninstalled pip-25.2
  WARNING: Failed to write executable - trying to use .deleteme logic
  Rolling back uninstall of pip
  Moving to c:\users\han\scoop\persist\python\lib\site-packages\pip-25.2.dist-info\
   from C:\Users\Han\scoop\persist\python\Lib\site-packages\~ip-25.2.dist-info
  Moving to c:\users\han\scoop\persist\python\lib\site-packages\pip\
   from C:\Users\Han\scoop\persist\python\Lib\site-packages\~ip
  Moving to c:\users\han\scoop\persist\python\scripts\
   from C:\Users\Han\scoop\persist\python\~cripts
ERROR: Could not install packages due to an OSError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Han\\scoop\\apps\\python\\current\\Scripts\\pip.exe' -> 'C:\\Users\\Han\\scoop\\apps\\python\\current\\Scripts\\pip.exe.deleteme'

Adding an empty file placeholder.txt as recommended by @mmdriley above works.

tanyanghan avatar Nov 25 '25 01:11 tanyanghan