repo2docker icon indicating copy to clipboard operation
repo2docker copied to clipboard

Add preBuild hook to configuration standard

Open ed-randall-blk opened this issue 10 months ago • 4 comments

Proposed change

Builds which follow the repo2docker configuration standard can be difficult to configure behind certain firewalled environments where a proxy must be configured to gain "outside" Internet access.

For example, (ref. pangeo-docker-images issue-517) the pangeo/base-image/Dockerfile contains a number of ONBUILD instructions (following repo2docker configuration standard) which are intended to help the developer with installs of additional packages etc, but none of these steps works; We see errors from the ONBUILD steps like:

    RuntimeError: Download error (60) SSL peer certificate or SSH remote key was not OK [https://conda.anaconda.org/conda-forge/noarch/repodata.json]
    SSL certificate problem: self-signed certificate in certificate chain

A preBuild step similar to the existing [postBuild](https://repo2docker.readthedocs.io/en/latest/config_files.html#postbuild-run-code-after-installing-the-environment) would be provide a useful point where we could configure the additional certs, proxy environment config, and anything else required before apt-get package installation can succeed.

Possible code for the preBuild hook (untested): (Insert at line 117 of https://github.com/pangeo-data/pangeo-docker-images/blob/master/base-image/Dockerfile between existing ONBUILD Checking for binder and Checking for apt.txt )

# If a preBuild file exists, run it
ONBUILD RUN echo "Checking for 'preBuild'..." \
        ; [ -d binder ] && cd binder \
        ; [ -d .binder ] && cd .binder \
        ; if test -f "preBuild" ; then \
        chmod +x preBuild \
        && ./preBuild \
        ; fi

Alternative options

As a workaround we have to forego the convenience of the provided ONBUILD steps and repeat the same logic in our own Dockerfile which builds FROM pangeo/base-image. This seems repetitive and should not be necessary.

We've considered forking pangeo/base-image into our own repo to make these changes possible but that's a highly undesirable option.

Who would use this feature?

Anyone who wishes to make a build following repo2docker configuration standards but is stuck behind a proxy firewall.

How much effort will adding it take?

Patch is above - I would estimate less than half a day for documentation updates and code integration.

Who can do this work?

Basic shell skills, ability to read and write documentation.

ed-randall-blk avatar Apr 03 '24 12:04 ed-randall-blk

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

welcome[bot] avatar Apr 03 '24 12:04 welcome[bot]

The r2d configuration files are designed to enabled reproducible repositories, but having a deployment specific pre-build breaks that, since it's only applicable to a single deployment and may actively break other deployments.

It sounds like you really want a deployment specific override separate from the repositories, e.g. in a global configuration file. repo2docker already has an appendix option https://github.com/jupyterhub/repo2docker/blob/d2f467cef603b5c80e5c9af5e7e794f724de703b/repo2docker/app.py#L234-L242 so maybe the equivalent for that, but at the start of the Dockerfile, could be useful, and you give all your users a standard repo2docker configuration file?

manics avatar Apr 06 '24 14:04 manics

Our intended use for the proposed step is simply setting 3 environment variables (probably in a profile file, since following steps should be able to pick them up) and installing an SSL cert. The problem we have is that, without setting these first, the subsequent steps cannot succeed. If pre-appendix (for want of a name) could do that, we're good.

ed-randall-blk avatar Apr 08 '24 11:04 ed-randall-blk

An equivalent for appendix but for the start of the Dockerfile would be perfect and should be sufficient to set up environment variables or other configs to make rep2docker work behind corporate firewalls. I'm also facing similar problems as @ed-randall-blk, using repo2docker behind a corporate firewall is a complete nightmare.

MartinK84 avatar May 17 '24 13:05 MartinK84