Injecting different type of credentials when installing private repositories

[rick2ricks]

When installing private repositories I would like to replace the authentication method between ssh and user password using environment variables as following:

"""
Setting environment variable to:
CREDENTIAL=https://user:pass
or
CREDENTIAL=ssh://git
"""

//Pipfile
[packages]
myapp = { git="${CREDENTIAL}@repo.com" }

But this will raise:

pipenv.patched.pip._vendor.packaging.requirements.InvalidRequirement: Invalid URL: git+${CRED}@repo.com

The only way I could get it to work was replacing the whole string:

#REPOSITORY=https://user:[email protected]
#REPOSITORY=ssh://[email protected]

//Pipfile
[packages]
myapp = { git="${REPOSITORY}" }

Am I missing something or this is the desired behaviour?

[matteius]

I'm glad you figured out a way to get it to work @rick2ricks -- ideally we can improve this to make it more resilient to different patterns. Also, In the case that you had it work, does the ${REPOSITORY} variable make it to the Pipfile.lock or are the credentials what is getting stored there?

[rick2ricks]

Hi, thanks for the response.

At my Pipfile.lock does not appear any credentials, it shows like the following:

  "mypackage": {
            "git": "${REPOSITORY}",
            "ref": "5654684646468464648646464",
            "subdirectory": "projects/myproject"
        },

But it would be a nice add if we could replace any part of the string.

Best regards.

[matteius]

Analysis of Pipenv Issue #6233

1. Problem Summary:

The issue highlights a problem with Pipenv's handling of environment variables within VCS URLs in the Pipfile. While using an environment variable to define the entire repository URL works, substituting only parts of the URL (e.g., just the username/password or just the protocol) results in an InvalidRequirement error. The user desires more fine-grained control over environment variable substitution within VCS URLs.

2. Comment Analysis:

• The maintainer acknowledges the issue and the user's workaround of substituting the entire URL.
• The user confirms that their workaround does not expose credentials in the Pipfile.lock, enhancing security.
• The user expresses a desire for more granular substitution of URL components.

3. Proposed Resolution:

The core issue lies in Pipenv's dependency parsing logic. It currently lacks the ability to recognize and handle environment variable placeholders within individual components of a VCS URL.

Here's a potential solution:

• Enhance URL Parsing: Modify the pipenv/utils/dependencies.py file, specifically the install_req_from_pipfile() function, to handle environment variable expansion within individual URL components before constructing the InstallRequirement object. This should involve:

  • Identifying placeholders like ${CREDENTIAL} within the URL.
  • Expanding those placeholders using os.path.expandvars().
  • Reconstructing the URL with the expanded values.

• Refactor Validation: Update the validation logic within install_req_from_pipfile() to accommodate the possibility of environment variables within URL components.

• Improve Error Handling: If an environment variable is not defined, provide a more informative error message, explicitly mentioning the undefined variable.

4. Code Snippet:

def install_req_from_pipfile(name, pipfile):
    # ... (Existing Code) ...

    if vcs:
        vcs_url = _pipfile[vcs]
        # Expand environment variables within the URL components
        vcs_url = os.path.expandvars(vcs_url)
        # ... (Rest of the code) ...

    # ... (Rest of the code) ...

5. Additional Steps:

• Comprehensive Testing: Implement tests to ensure this new functionality works correctly across different VCS types, URL formats, and potential edge cases involving undefined environment variables.
• Documentation: Update Pipenv's documentation to reflect this new feature and provide usage examples.
• Consider Alternative Solutions: While environment variable substitution provides flexibility, explore alternative approaches like a separate configuration file for sensitive information or support for credential helpers for a more robust and secure solution.

This issue highlights the need for Pipenv to handle environment variables more intelligently within Pipfile, providing users with greater flexibility while maintaining security and clarity.

==================================================

[matteius]

@rick2ricks could you check if this PR fixes it: https://github.com/pypa/pipenv/pull/6276