harbor-helm Cannot connect to registry behind HTTP proxy

When installing the chart version 1.5.3 from the goharbor Helm repo the HTTP proxy does not work, e.g. when trying to add a new registry endpoint and test the connection. Also doing a curl on the harbor-core pod fails.

It appears that this Harbor Helm chart is defining the *_PROXY environment variables that are used in several components in uppercase. However, when defining the *_PROXY environment variables in lowercase as e.g. http_proxy the proxy works. Both in the CLI and when trying to add a registry and pull some images. It appears that there is no standard way to define the proxy environment variables.

I tested this with a modified helm chart that I will attach as pull request. Please let me know if it can be merged or if there is something I can change on my side.

Simple example:

harbor [ /harbor ]$ env | grep HTTP_PROXY
HTTP_PROXY=http://my_company_proxy.com
harbor [ /harbor ]$ curl google.com
curl: (7) Failed to connect to google.com port 80: Connection refused
harbor [ /harbor ]$ export http_proxy=http://my_company_proxy.com
harbor [ /harbor ]$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

Feb 25 '21 08:02 carolili

I'm successfully using harbor behind a proxy using the proxy settings built in to this chart: https://github.com/goharbor/harbor-helm/blob/master/values.yaml#L349.

Using it to fetch images from dockerhub, quay and gcr all through proxy.

Edit: Using the 1.5.3 version of the chart.

Feb 25 '21 08:02 alxbse

@alxbse Thank you for the comment. Those settings do not work for me when the environment variables are uppercase. Would it be possible that there is something else in your environment that sets the lowercase version of the proxy variables?

Could you say what version of Kubernetes you are using and what container runtime + version you are using? I have the following:

Kubernetes: 1.19.7
Docker: 19.03

Perhaps it could be an issue stemming from Docker as container runtime?

Feb 25 '21 09:02 carolili

This is an interesting issue. Exec;ed into one of my core pods to verify and there are no lower case environment variables.

$ export | grep -i proxy
declare -x HTTPS_PROXY="http://corporate.proxy:3128"
declare -x HTTP_PROXY="http://corporate.proxy:3128"
declare -x NO_PROXY="harbor-harbor-core,harbor-harbor-jobservice,harbor-harbor-database,harbor-harbor-chartmuseum,harbor-harbor-clair,harbor-harbor-notary-server,harbor-harbor-notary-signer,harbor-harbor-registry,harbor-harbor-portal,harbor-harbor-trivy,127.0.0.1,localhost,.local,.internal"
declare -x PERMITTED_REGISTRY_TYPES_FOR_PROXY_CACHE="docker-hub,harbor"

Even with just uppercase environment variables I can use curl in the pod to reach docker hub.

It should not really matter but to verify with cluster info. We are using cri-o as the container runtime as a difference.

kubernetes: 1.19.1
cri-o: 1.19.0

Could it maybe be that you have a NO_PROXY variable that catches the traffic?

Feb 25 '21 10:02 alxbse

Interesting, it shouldn't be an issue with NO_PROXY as it only contains some default values. Also, when setting the lowercase version of the environment variables the connection works. Without changing the NO_PROXY. 🤔

Feb 25 '21 10:02 carolili

the upper case should work please use curl -v to figure out why

Mar 04 '21 08:03 reasonerjt

@carolili It may be related to you can not access corporate.proxy inside the container

Sep 08 '21 09:09 maxisacoder

I'm successfully using harbor behind a proxy using the proxy settings built in to this chart: https://github.com/goharbor/harbor-helm/blob/master/values.yaml#L349.

i can confirm that it started to work after adding UPPER-case proxy vars to the harbor.yml

Using it to fetch images from dockerhub, quay and gcr all through proxy.

Edit: Using the 1.5.3 version of the chart.

[...]
 Config http proxy for components, e.g. http://my.proxy.com:3128
# Components doesn't need to connect to each others via http proxy.
# Remove component from `components` array if want disable proxy
# for it. If you want use proxy for replication, MUST enable proxy
# for core and jobservice, and set `http_proxy` and `https_proxy`.
# Add domain to the `no_proxy` field, when you want disable proxy
# for some special registry.
proxy:
  http_proxy: http://my.proxy.com:3128
  HTTP_PROXY: http://my.proxy.com:3128
  https_proxy: http://my.proxy.com:3128
  HTTPS_PROXY: http://my.proxy.com:3128
  no_proxy: 127.0.0.1,localhost,.local,.internal
  NO_PROXY: 127.0.0.1,localhost,.local,.internal
  components:
   - core
   - jobservice
   - trivy
  [...]

Dec 29 '23 16:12 schonma

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

Feb 28 '24 09:02 github-actions[bot]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

Mar 30 '24 09:03 github-actions[bot]