Recommended install command broken in newer Python versions

[mcandre]

Issue description

The recommended way to install pipenv breaks on newer Python versions. For example, Ubuntu's pip now bans installing packages (both globally as well as by --user).

https://pipenv.pypa.io/en/latest/

Also, modern Python pip installations tend to name the application pip3 instead of pip, often reserving the latter for use with python v2.

Expected result

The recommended command for installing pipenv should reflect current best practices for Python package management.

Actual result

Ubuntu 24.04 in GitHub Actions shows installation of pipenv failing:

$ pip install --user pipenv
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Steps to replicate

1. Run the recommended command for acquiring pipenv in Ubuntu 24.04.

Workaround

Although this overcomplicates dev environment setup, planning to move pipenv provisioning from my usual platform agnostic requirements.txt requirements-dev.txt configuration files to platform specific commands.

However, installing pip packages outside of infrastructure as code files tends to restrict the ability of SCA tools to collect complete and accurate SBOM data for security scans.

Notes

If only Python (pip) would include pipenv by default, as it does with pipx.

[matteius]

Analysis for Issue #6264:

Analysis of Pipenv Issue #6264

1. Summary of the Problem:

The issue highlights two problems with Pipenv's recommended installation command:

• Outdated Practices: The command pip install --user pipenv relies on outdated practices for Python package management. Modern Python versions, especially on Linux distributions like Ubuntu 24.04, discourage global and user-level package installations in favor of virtual environments due to potential conflicts and system instability.
• Inaccurate Command: The command assumes the pip executable is named pip, which is not always true. Modern Python installations often use pip3 for Python 3 and reserve pip for Python 2.

These issues lead to the recommended installation command failing on newer Python versions and environments.

2. Analysis of the Discussion:

There are no comments on this issue yet, so there's no discussion to analyze.

3. Proposed Resolution:

The resolution should address both aspects of the problem:

• Update Documentation: The Pipenv documentation should be updated to reflect current best practices. The recommended installation command should guide users towards creating a virtual environment first, then installing Pipenv within that environment.
• Provide Alternative Commands: The documentation should offer alternative commands for different scenarios, including:
  • Installing Pipenv in a pre-existing virtual environment.
  • Installing Pipenv using pip3 for modern Python installations.
• Consider pipx: The issue suggests that Python (pip) should include Pipenv by default, as it does with pipx. While this is an interesting proposal, it might be out of scope for this specific issue. It could be a separate feature request or discussion.

4. Code Snippet:

Here's a potential code snippet to be added to the installation documentation:

## Installing Pipenv

**Recommended Method (for new projects):**

1. **Create a virtual environment:**
   ```bash
   python3 -m venv .venv
   ```
2. **Activate the virtual environment:**
   ```bash
   source .venv/bin/activate  # Linux/macOS
   .venv\Scripts\activate.bat  # Windows
   ```
3. **Install Pipenv within the virtual environment:**
   ```bash
   pip install pipenv
   ```

**Alternative Methods:**

* **In a pre-existing virtual environment:**
+422.4s  635    pip install pipenv
   ```
* **Using `pip3`:**
   ```bash
   pip3 install pipenv  # If `pip` is reserved for Python 2
   ```

5. Additional Steps/Investigations:

• Usability Testing: Conduct usability testing to ensure the updated documentation is clear and easy to follow for users with different levels of experience.
• Consider ensurepip: Pipenv could potentially use the ensurepip module to bootstrap pip within the virtual environment if it's not already present, simplifying the process further.
• Investigate pipx Integration: Explore the feasibility and benefits of including Pipenv as a default package within Python's pipx. This could be a long-term solution to streamline the installation process.

This issue presents an opportunity to improve Pipenv's user experience, especially for new users. By addressing the outdated practices and providing clear instructions, Pipenv can become more accessible and user-friendly.

[nbrew]

Encountered this last week and arrived at the same set of steps @matteius suggested in the Recommended Method in the code snippet above. +1 for that addition.

[kdickerson]

I find the solution of first creating and activating a virtual environment to use pipenv extremely awkward. If I have to activate a venv in order to use pipenv it loses a significant chunk of the convenience of using pipenv to manage my virtual environments for me. What's the remaining value proposition of pipenv compared to any other tooling for dealing with dependencies if I have to manage a virtual environment manually anyway?

I fully admit I don't have a better solution and do not have a strong grasp of the underlying reasons given in PEP 668 for why pip --user is no longer considered a viable solution.

I just want to prod in hopes that someone will think of something clever. A big draw of pipenv for me is that using it is "no friction" across projects. That is, I can have a dozen projects using pipenv and I work with them all the same without cognitive overhead. When I'm working in the project I prefix my commands with pipenv and it "just works." It seems like we're losing that.

Would a viable approach to keeping that "no friction" be to create a "user level" venv that holds pipenv that I could activate automatically in my bash profile? Other thoughts on how to keep this as convenient as it currently is?

[kdickerson]

I got back around to checking on this for my new development environment. I'm trying out a "user level" venv as I proposed above. Seems to be working so far (assuming you use a distribution that uses .profile, otherwise adjust as needed):

$ python3 -m venv ~/.user-venv
$ echo -e '\n\n# Activate isolated Python venv\nexport PIPENV_IGNORE_VIRTUALENVS=1' >> ~/.profile
$ echo -e 'source "$HOME/.user-venv/bin/activate"' >> ~/.profile
$ source ~/.profile

When activated interactively the venv will update your bash prompt to indicate its in the venv, but when it activates "automatically" at login it doesn't. So it more or less does what I need.

Thought I'd throw this out there as an approach that (seems to) put me back at the level of convenience from before the PEP-668 changes.