pipenv icon indicating copy to clipboard operation
pipenv copied to clipboard
verified publisher icon pypa

variable expansion doesnt work inside Pipfile in order to install from a private repo

Open Tlaloc-Es opened this issue 3 years ago • 2 comments

Issue description

I am trying to install a dependency from a private repo in order to this I have the following line

data = {editable = true, ref = "branch", git="https://${USER}:${ACCESS_TOKEN}@gitlab.com/private/repo"}

If I hardcode USER and ACCESS_TOKEN this work perfectly but not with shell expansion in order to pick the vairbles

Expected result

install packages and keep the env expansion inside the Pipfile and Pipfile.lock

Actual result

I cannot install from private repo without hardcode USER and ACCESS_TOKEN

Steps to replicate

Create a private repo in gitlab or github Create a env variables with your user and password Try to install that repository

$ pipenv --support

Pipenv version: '2022.6.7'

Pipenv location: '/home/user/a/.venv/lib/python3.9/site-packages/pipenv'

Python location: '/home/user/a/.venv/bin/python'

Python installations found:

  • 3.9.12: /home/user/a/.venv/bin/python3.9
  • 3.9.12: /home/user/a/.venv/bin/python
  • 3.9.12: /home/user/a/.venv/bin/python3
  • 3.9.12: /home/user/a/.venv/bin/python3.9
  • 3.9.12: /home/user/a/.venv/bin/python
  • 3.9.12: /home/user/a/.venv/bin/python3
  • 3.9.12: /home/user/anaconda3/bin/python3.9
  • 3.9.12: /home/user/anaconda3/bin/python
  • 3.9.12: /home/user/anaconda3/bin/python3
  • 3.8.13: /home/user/.pyenv/versions/3.8.13/bin/python
  • 3.8.10: /home/user/.pyenv/versions/3.8.10/bin/python
  • 3.8.10: /usr/bin/python3
  • 3.8.10: /usr/bin/python3.8
  • 3.8.10: /bin/python3
  • 3.8.10: /bin/python3.8
  • 3.6.15: /home/user/.pyenv/versions/3.6.15/bin/python3.6
  • 3.6.0: /home/user/.pyenv/versions/3.6.0/bin/python3.6
  • 2.7.18: /usr/bin/python2
  • 2.7.18: /usr/bin/python
  • 2.7.18: /usr/bin/python2.7
  • 2.7.18: /bin/python2
  • 2.7.18: /bin/python
  • 2.7.18: /bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.12',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.13.0-52-generic',
 'platform_system': 'Linux',
 'platform_version': '#59~20.04.1-Ubuntu SMP Thu Jun 16 21:21:28 UTC 2022',
 'python_full_version': '3.9.12',
 'python_version': '3.9',
 'sys_platform': 'linux'}

System environment variables:

  • SHELL
  • SESSION_MANAGER
  • QT_ACCESSIBILITY
  • COLORTERM
  • XDG_CONFIG_DIRS
  • NVM_INC
  • XDG_MENU_PREFIX
  • TERM_PROGRAM_VERSION
  • GNOME_DESKTOP_SESSION_ID
  • APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL
  • GNOME_SHELL_SESSION_MODE
  • SSH_AUTH_SOCK
  • XMODIFIERS
  • DESKTOP_SESSION
  • SSH_AGENT_PID
  • NO_AT_BRIDGE
  • GTK_MODULES
  • DBUS_STARTER_BUS_TYPE
  • PWD
  • KRB5CCNAME
  • XDG_SESSION_DESKTOP
  • LOGNAME
  • XDG_SESSION_TYPE
  • GPG_AGENT_INFO
  • XAUTHORITY
  • VSCODE_GIT_ASKPASS_NODE
  • GJS_DEBUG_TOPICS
  • WINDOWPATH
  • HOME
  • USERNAME
  • IM_CONFIG_PHASE
  • LANG
  • LS_COLORS
  • XDG_CURRENT_DESKTOP
  • VTE_VERSION
  • GIT_ASKPASS
  • GNOME_TERMINAL_SCREEN
  • INVOCATION_ID
  • MANAGERPID
  • CHROME_DESKTOP
  • GJS_DEBUG_OUTPUT
  • NVM_DIR
  • VSCODE_GIT_ASKPASS_EXTRA_ARGS
  • LESSCLOSE
  • XDG_SESSION_CLASS
  • TERM
  • LESSOPEN
  • LIBVIRT_DEFAULT_URI
  • USER
  • VSCODE_GIT_IPC_HANDLE
  • GNOME_TERMINAL_SERVICE
  • DISPLAY
  • SHLVL
  • NVM_CD_FLAGS
  • QT_IM_MODULE
  • DBUS_STARTER_ADDRESS
  • XDG_RUNTIME_DIR
  • PYENV_ROOT
  • VSCODE_GIT_ASKPASS_MAIN
  • JOURNAL_STREAM
  • XDG_DATA_DIRS
  • GDK_BACKEND
  • PATH
  • GDMSESSION
  • ORIGINAL_XDG_CURRENT_DESKTOP
  • DBUS_SESSION_BUS_ADDRESS
  • NVM_BIN
  • GIO_LAUNCHED_DESKTOP_FILE_PID
  • GIO_LAUNCHED_DESKTOP_FILE
  • TERM_PROGRAM
  • _
  • OLDPWD
  • P9K_TTY
  • _P9K_TTY
  • ZSH
  • PAGER
  • LESS
  • LSCOLORS
  • P9K_SSH
  • NOMAD_ADDR
  • NOMAD_CLIENT_CERT
  • NOMAD_CLIENT_KEY
  • NOMAD_CACERT
  • NOMAD_TOKEN
  • CONSUL_HTTP_ADDR
  • CONSUL_HTTP_SSL
  • CONSUL_HTTP_SSL_VERIFY
  • CONSUL_CLIENT_CERT
  • CONSUL_CLIENT_KEY
  • CONSUL_CACERT
  • CONSUL_HTTP_TOKEN
  • KAFKA_BIN
  • KAFKA_CONFIG
  • GOPATH
  • PYENV_SHELL
  • PIPENV_VENV_IN_PROJECT
  • buildtime_USER_ACCESS_TOKEN
  • buildtime_ACCESS_TOKEN
  • CONDA_EXE
  • _CE_M
  • _CE_CONDA
  • CONDA_PYTHON_EXE
  • CONDA_SHLVL
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_PYTHON_PATH
  • PIPENV_ACTIVE
  • CONDA_PREFIX
  • CONDA_DEFAULT_ENV
  • CONDA_PROMPT_MODIFIER
  • VIRTUAL_ENV
  • PS1
  • PIP_SHIMS_BASE_MODULE
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

  • PIPENV_VENV_IN_PROJECT: enabled
  • PIPENV_ACTIVE: 1

Debug–specific environment variables:

  • PATH: /home/user/a/.venv/bin:/home/user/bin:/home/user/.local/bin:/usr/local/bin:/home/user/anaconda3/bin:/home/user/Software/Kafka/bin:/home/user/.nvm/versions/node/v16.14.0/bin:/home/user/bin:/home/user/.local/bin:/usr/local/bin:/home/user/anaconda3/condabin:/home/user/Software/Kafka/bin:/home/user/.nvm/versions/node/v16.14.0/bin:/home/user/.poetry/bin:/home/user/.pyenv/shims:/home/user/.pyenv/bin:/home/user/.local/bin:/home/user/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/user/.fzf/bin:/home/user/bin:/home/user/bin:/usr/local/go/bin:/home/user/go/bin:/home/user/bin:/home/user/anaconda3/bin:/usr/local/go/bin:/home/user/go/bin:/home/user/bin:/home/user/anaconda3/bin
  • SHELL: /bin/bash
  • LANG: es_ES.UTF-8
  • PWD: /home/user/a
  • VIRTUAL_ENV: /home/user/a/.venv

Contents of Pipfile ('/home/user/a/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
data = {editable = true, ref = "branch", git="https://${USER}:'${buildtime_ACCESS_TOKEN}'@gitlab.com/private/repo"}

[dev-packages]
pytest = "*"

[requires]
python_version = "3.9"

[scripts]
start = "python src/main.py"
test = "pytest -s"

Tlaloc-Es avatar Jun 29 '22 10:06 Tlaloc-Es

I was not able to reproduce this issue. You can see in my output that it expanded the credentials I exported into the shell ahead of time. I didn't specify a real private repo to test this, but its apparent that the variables were expanded for me. @Tlaloc-Es Can you elaborate on your system and the steps you took?

(pipenv-5153) matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5153$ export USER=test

(pipenv-5153) matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5153$ export ACCESS_TOKEN=test

(pipenv-5153) matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5153$ pipenv lock -v
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
ERROR:pip.subprocessor:[present-diagnostic] git clone --filter=blob:none 'https://test:****@gitlab.com/private/repo' /tmp/pipenv-ayxamgxu-src/data exited with 128
....
pipenv.patched.notpip._internal.exceptions.InstallationSubprocessError: git clone --filter=blob:none 'https://test:****@gitlab.com/private/repo' /tmp/pipenv-ayxamgxu-src/data exited with 128
✘ Locking Failed! 

(pipenv-5153) matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5153$ ls
Pipfile
(pipenv-5153) matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5153$ cat Pipfile 
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
data = {editable = true, ref = "branch", git="https://${USER}:${ACCESS_TOKEN}@gitlab.com/private/repo"}

[dev-packages]

[requires]
python_version = "3.10"

matteius avatar Jul 06 '22 05:07 matteius

@Tlaloc-Es Can you check with pipenv=2022.8.31?

matteius avatar Sep 01 '22 12:09 matteius