proxying pip (HTTPS/SSL)

[hoijui]

trafficstars

As a reference to others (and possibly myself), i explain how it worked for me.

i run this docker (docker-proxy) like this:

sudo docker build -t docker-proxy .
./run.sh ssl

Then i copy test/detect-proxy.sh to the root directory of my own docker containers sources.

My own docker containers Dockerfile looks like this:

# Base image
FROM python:2-slim

MAINTAINER me <[email protected]>

WORKDIR /src

# We need info about available system packages
RUN apt-get update

# These are required by detect-proxy.sh
RUN apt-get install -y --no-install-recommends ca-certificates net-tools netcat

ADD ./detect-proxy.sh /src/detect-proxy.sh
RUN /src/detect-proxy.sh

# These are required by one of our python dependencies
RUN apt-get install -y gcc libreadline-dev

# Install python requirements
# ... option 1 (final solution):
ADD ./requirements.txt /src/requirements.txt
RUN pip --cert /etc/ssl/certs/ca-certificates.crt install -r requirements.txt
# option 2 (may make sense during development):
RUN pip --cert /etc/ssl/certs/ca-certificates.crt install numpy
RUN pip --cert /etc/ssl/certs/ca-certificates.crt install enum34

# Execute the python script
CMD ["python", "/src/my_script.py"]

This way, all downloads (at least by apt-get and pip) done while building and running the docker, go through the proxy. One thing that might be clear anyway, but worth to note: The proxy cache is lost whenever we shut the docker-proxy container down (with Ctrl+C).

Is there a way to prevent that? In other words, can we keep/carry over proxy cache between different runs of ./run.sh?

[silarsis]

You can but it's a bit of stuffing around - essentially the squid cache directory would need to be externalised - either to local disk or to another container. ISTR I did that early on in the piece, then removed it later because it was too many moving parts.

[tomhatzer]

@hoijui you could also save the cache into a volume by providing the following parameters to docker run: --volume /srv/docker/squid/cache:/var/spool/squid3 (customize the cache path with your preferred local path)