chalice package doesn't handle git repos referenced in requirements.txt properly

[skwashd]

Description

When using chalice package to package my app I expect it to respect the git repos referenced in the requirements.txt file. Instead it uses the version of the installed package to fetch a wheel. This makes it impossible to reference commits in unreleased versions or forks of published python packages.

Steps To Reproduce

1. Create a requirements.txt file that includes a reference to a git repo (see end of this bug report for an example)
2. Run chalice package
3. Extract the zip file
4. Check the contents of github3/repos/commit.py
5. Notice that the associated_pull_requests method isn't there
6. Swear

This was tested with Chalice 1.18.1 on Python 3.7 with the build running on CodeBuild

Expected Result

Instead of pulling the wheel, chalice will pull the code from the git repo as instructed by the requirements.txt file. This would also result in the associated_pull_requests method being present in github3/repos/commit.py

requirements.txt

-e git+ssh://[email protected]/sigmavirus24/github3.py@197173b5669b189f94c433261497cb652f1db1a9#egg=github3
semantic-version==2.8.5

[jamesls]

Confirmed bug.

What's happening here is we initially download all the packages from requirements.txt (which will correctly pull from the git repo/commit) and get a list of all packages that didn't download as a wheel file. We then try to search pypi for a corresponding wheel based on the package name and version. The github3.py package downloads as a zip file because it's pointing at a git repo so we download the .whl file from pypi. There doesn't seem to be anything in the github3-py-1.3.0.zip file that indicates it was downloaded as an editable file (other than checking for a .git/ repository which is specific to the VCS backend). I think the best option is to parse the requirements.txt file and try to mark which packages are editable and special case them as needed.

[skwashd]

@jamesls thanks for the confirmation. Is this something you are likely to fix in the short term or should I try to get someone from my team to look at a fix?

[subssn21]

I don't think this fix solves the problem, tried checking it out locally and doesn't solve the problem for me. I can't under any circumstances get a git repo to install and add the files to the .zip file I have tried:

file:///Users/dramus/gather#egg=gather --no-index -e git+git:[email protected]:gatherthefamily/gather.git@main#egg=gather (Gives a package not found error) -e git+ssh:[email protected]:gatherthefamily/gather.git@main#egg=gather (Gives a package not found error)

[Brianthered]

https://github.com/aws/chalice/issues/1700#issuecomment-829703418 does a good job of describing this problem step by step. Cross linking to point similar issues out.

[Brianthered]

FYI, I ended up dropping the packages I needed into /vendor to work around this problem similar to what was done in ticket 1700, Mentioning that here so maybe the next person doesn't have to poke around as much.