pip differ doesn't match the packages with python version suffix in name

[michalvanco]

Consider image with python-3.6 inside and additional pip packages installed via requirements file (via setup.py).

your app includes requirements.txt with following packages:

graphitesend
future

When you perform container-diff analyze daemon://your-image -t pip you receive only subset of pip packages:

NAME               VERSION        SIZE          INSTALLATION
-pip               18.0           6.8M          /usr/local/lib/python3.6/site-packages
-pytz              2018.9         954.9K        /usr/local/lib/python3.6/site-packages
-setuptools        39.0.1         1.8M          /usr/local/lib/python3.6/site-packages
-six               1.12.0         31.7K         /usr/local/lib/python3.6/site-packages

While inside the image you can see following packages:

$ ls /usr/local/lib/python3.6/site-packages/
graphitesend
graphitesend-0.10.0-py3.6.egg-info
future
future-0.16.0-py3.6.egg-info                  
pip
pip-18.0.dist-info
pytz
pytz-2018.9.dist-info
setuptools
setuptools-39.0.1.dist-info
six.py
six-1.12.0.dist-info

$ pip list
Package                 Version
----------------------- -------
future                  0.16.0 
graphitesend            0.10.0 
pip                     18.0   
pytz                    2018.9 
setuptools              39.0.1 
six                     1.12.0 

I suspect following piece of code in differ: packageDir := regexp.MustCompile("^([a-z|A-Z|0-9|_]+)-(([0-9]+?\\.){2,3})(dist-info|egg-info)$") which doesn't match these packages because something like -py3.6 is not allowed in the filename.

Expected behavior

All packages from pip list are listed.

Actual behavior

Only subset of pip packages are listed in differ output.

Information

• container-diff version: version 0.14.0
• Operating system: CentOS 7.5

Thank you in advance for any update!

[nkubala]

@michalvanco thanks for the issue! There are a few issues with the pip analysis right now that I'm working through, I'll try and get to this ASAP. Would you mind giving me the Dockerfile you used to create an image that I can repro this issue with? Thanks!

[michalvanco]

Hi Nick, thanks for a quick update. I'll prepare a simplified Dockerfile which you can use for the test today or tomorrow.

[nkubala]

@michalvanco I think https://github.com/GoogleContainerTools/container-diff/pull/292 might address this. Once it's merged can you build from HEAD and see if it fixes your issue?

[michalvanco]

Hi Nick,

here's the example dockerfile to reproduce the issue.

requirements.txt

graphitesend

Dockerfile

FROM centos/python-36-centos7:latest
WORKDIR /test-dir
COPY requirements.txt /test-dir
RUN pip3.6 install -r requirements.txt

after docker build you will find the following content inside

$ ls -la /opt/app-root/lib64/python3.6/site-packages/
total 52
drwxrwxr-x  1 default root 4096 Jan 28 10:00 .
drwxrwxr-x  1 default root 4096 Jan 17 08:06 ..
-rw-rw-r--  1 default root  126 Jan 17 08:06 easy_install.py
drwxr-xr-x  3 default root 4096 Jan 28 10:00 graphitesend
drwxr-xr-x  2 default root 4096 Jan 28 10:00 graphitesend-0.10.0-py3.6.egg-info
drwxrwxr-x 11 default root 4096 Jan 17 08:06 pip
drwxrwxr-x  2 default root 4096 Jan 17 08:06 pip-9.0.1.dist-info
drwxrwxr-x  5 default root 4096 Jan 17 08:06 pkg_resources
drwxrwxr-x  2 default root 4096 Jan 17 08:06 __pycache__
drwxrwxr-x  5 default root 4096 Jan 17 08:06 setuptools
drwxrwxr-x  2 default root 4096 Jan 17 08:06 setuptools-28.8.0.dist-info
drwxrwxr-x  6 default root 4096 Jan 17 08:06 wheel
drwxrwxr-x  2 default root 4096 Jan 17 08:06 wheel-0.29.0.dist-info

Let me know if any part is not clear, thanks!