AWS CodeArtifact login does not find pip package manager when Python 3 was installed from Homebrew

[oura-kataja]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone though the User Guide and the API reference
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug AWS CodeArtifact login does not find pip package manager when Python 3 was installed from Homebrew. This installs current Python 3 to python3 while legacy Python 2 remains at python. Python package manager install pip for Python 3 as pip3 reflects this.

SDK version number aws-cli/2.2.31 Python/3.9.6 Darwin/20.5.0 source/x86_64 prompt/off

Platform/OS/Hardware/Device macOS Big Sur 11.4 (20F71) Homebrew 3.2.9 [email protected]: stable 3.9.6 (bottled) pip 21.2.4 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

To Reproduce (observed behavior) Login to CodeArtifact repository by using aws codeartifact login --tool pip

Expected behavior The tool updates pip configuration file with CodeArtifact credentials.

Logs/output

$ aws codeartifact login --tool pip ... removed repository and AWS account specific information ...
pip was not found. Please verify installation.

Additional context Workaround to the problem is to install another pip with itself using: pip3 install pip

[kdaily]

Hi @oura-kataja,

Sorry to hear you're having an issue. I'm looking into this.

~Of note, AWS CLI v2 uses it's own Python interpreter, not one present on the system. Can you confirm how you're installing the AWS CLI v2 as well? Thanks!~

This occurs with AWS CLI v1 as well. It's a known Homebrew limitation. One workaround is to symlink the pip3 executable to pip, and then add that directory to your PATH:

❯ pwd
/Users/kdaily/.local/bin
❯ ln -s `which pip3` pip
❯ export PATH=/Users/kdaily/.local/bin:$PATH
❯ which pip
/Users/kdaily/.local/bin/pip
❯ pip --version
pip 21.1.3 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
❯ aws codeartifact login --tool pip --repository myrepo --domain mydomain --domain-owner 000000000000
Successfully configured pip to use AWS CodeArtifact repository https://mydomain-968989450063.d.codeartifact.us-west-2.amazonaws.com/pypi/myrepo/
Login expires in 12 hours at 2021-08-26 01:28:37-07:00

I'll talk with the CodeArtifact team if any change would be made to account for this.

[PeterMalkin]

Still happening. The workaround does not help

[hadoop@ip-172-31-43-32 ~]$ aws codeartifact login --tool pip --repository ceti --domain ceti-repo pip was not found. Please verify installation.

[hadoop@ip-172-31-43-32 ~]$ pip3 install pip Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: pip in /usr/lib/python3.7/site-packages (20.2.2)

[hadoop@ip-172-31-43-32 ~]$ pip --version pip 20.2.2 from /usr/lib/python3.7/site-packages/pip (python 3.7)

[hadoop@ip-172-31-43-32 ~]$ aws codeartifact login --tool pip --repository ceti --domain ceti-repo pip was not found. Please verify installation.

[datacubed]

Same is happening here

[OlivierCloudar]

I have the exact same output, but on another system:

aws-cli/2.4.6 Python/3.8.8 Windows/10 exe/AMD64 prompt/off Windows 11 Chocolatey 0.11.3 (used to install pyenv-win) pyenv 2.64.11 pip 21.3.1 from c:\users\user.pyenv\pyenv-win\versions\3.8.10\lib\site-packages\pip (python 3.8)

The thing in common does seem to be that I did not install python/pip using the default packages either, but using pyenv.

 ~   aws codeartifact login --tool pip --domain domainxxxx --repository myrepooo --debug

..left out due to sensitive information...
2022-01-10 23:49:55,204 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli\customizations\codeartifact\login.py", line 88, in _run_commands
  File "subprocess.py", line 493, in run
  File "subprocess.py", line 858, in __init__
  File "subprocess.py", line 1311, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "awscli\clidriver.py", line 459, in main
  File "awscli\clidriver.py", line 594, in __call__
  File "awscli\customizations\commands.py", line 191, in __call__
  File "awscli\customizations\codeartifact\login.py", line 663, in _run_main
  File "awscli\customizations\codeartifact\login.py", line 378, in login
  File "awscli\customizations\codeartifact\login.py", line 95, in _run_commands
ValueError: pip was not found. Please verify installation.

pip was not found. Please verify installation.

[richardj-bsquare]

Yes, I have this problem; OSX intercepts the invocation of pip and tries to get you to install python2.x via xcode-select...

~/.local/bin 🐏  13GiB/16GiB(9GiB/10GiB) 
at 12:57:03 PM 🍺 ls -l pip*
lrwxr-xr-x richardj staff 13 B Tue Jan 18 12:48:12 2022  pip ⇒ /usr/bin/pip3
lrwxr-xr-x richardj staff 13 B Tue Jan 18 12:54:27 2022  pip3 ⇒ /usr/bin/pip3

~/.local/bin 🐏  13GiB/16GiB(9GiB/10GiB) 
at 12:57:08 PM 🍺 ./pip3 --version
pip 21.3.1 from /Users/richardj/Library/Python/3.8/lib/python/site-packages/pip (python 3.8)

~/.local/bin 🐏  13GiB/16GiB(9GiB/10GiB) 
at 12:57:18 PM 🍺 ./pip --version 
xcode-select: Failed to locate 'pip', requesting installation of command line developer tools.

The only way I could get around this was to install python3 via brew and point pip at that instead...

~/.local/bin 🐏  13GiB/16GiB(9GiB/10GiB) 
at 01:00:52 PM 🔥 ln -s /opt/homebrew/Cellar/[email protected]/3.10.1/bin/pip3 pip

~/.local/bin 🐏  13GiB/16GiB(9GiB/10GiB) 
at 01:00:57 PM 🍺 ./pip --version
pip 21.3.1 from /opt/homebrew/lib/python3.10/site-packages/pip (python 3.10)

Please help us out by making a --tool=pip3 option as well as --tool=pip

[michaelwang11394]

@kdaily Any update? Workaround does not work.

[kdaily]

Hi all, I don't have any update on a fix for this. I hear that not being able to use a specific pip executable is problematic. The workaround I posted above (for Mac specifically) still works for me. I'm also able to use that within a venv too, but still on Mac, not on Windows.

I understand that it's also not ideal to not be able to use the idiomatic solution, but the CodeArtifact documentation gives another way to do this, but I'm not familiar enough with the usage to know if that is workable past configuration.

https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-without-pip.html

[kdaily]

Hi all, the CodeArtifact team is aware of this limitation and is exploring options to improve this. I don't have any timelines to share as to when a change would be available. I'll update as I learn more.

[aJetHorn]

The workaround does work but I'm still kicking myself every time I end up back on this page

[PeterMalkin]

I had to change my boostrap script to the following to work around the issue:

/usr/bin/python3 -m pip install pip --upgrade --no-warn-script-location PATH=$PATH:/home/hadoop/.local/bin aws codeartifact login --tool pip --repository myrepository --domain myrepository-domain /home/hadoop/.local/bin/pip3 install --no-warn-script-location myrepository sudo ln -s /home/hadoop/.local/bin/myrepository /usr/bin/myrepo

[stutibiyani]

Hi all, this error was fixed for me by creating a virtual environment and then executing the aws codeartifact login --tool pip command.

[cameron-headspace]

Worked with homebrew brew instal python. Output from install:

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/[email protected]/libexec/bin

Just add /usr/local/opt/[email protected]/libexec/bin to path.

[richardj-bsquare]

To do the previous post suggestion in an automatic way on your shell for python3 on OSX with homebrew, add this to your .bashrc/.zshrc (assumes you have sed available)...

export PYTHON3_VER=$(python3 --version | sed 's/Python //g')
export PYTHON3_MAJOR_VER=$(echo "$PYTHON3_VER" | sed 's/\.[0-9]*$//g')

export PATH="/opt/homebrew/Cellar/python@${PYTHON3_MAJOR_VER}/${PYTHON3_VER}/libexec/bin:$PATH"

This can be adapted if you have a different location for your python3.

[markoraiha]

Found easy workaround for this issue:

aws codeartifact login --tool pip --domain my-domain --repository my_repo --region eu-west-1 --dry-run | sed 's/^pip /pip3 /'

[rahul-ve]

I tried the suggested workaround but it did not work. What worked was to make sure the conda env I was using had the same Python version used by AWS CLI, which was 3.9 on my box. Hope this helps someone!

[TannerBaldus]

Kept coming back to this thread for @markoraiha so made a convenience bash function based on it.

https://gist.github.com/TannerBaldus/39f774369e4ca4965832ef4b7855b2f3

You can copy this into your ~/.zshrc or ~/.bashrc and use it as documented. I preferred using it in an alias like so

alias aws-ca="code_artifact_login mDomain Owner123 MyRepo us-west-2 myProfile"

[gh-andre]

Still not fixed in 2023. Using venv worked for me.

python3 -m venv abc-venv
source abc-venv/bin/activate
aws codeartifact login --tool pip ...