Feature/dynamic upstream config for non resolveable hosts

[hishamanver]

This PR defines a new env var 'UPSTREAM_MAPPINGS' which is used to dynamically populate the nginx config with upstream -> server definitions.

This allows hosts that are not resolveable via resolvers to be reached. SImilar to how an /etc/hosts entry works, however this includes the benefits of round-robin

• trailing whitespaces also removed from readme (in addition to additional documentation for the new var)

[rpardini]

Interesting, apart from all the whitespace changes in the README. Could you explain a bit what is your use case for this?

[hishamanver]

Apologies for the whitespace stuff - I have 'trim trailing whitespace' enabled on my visual studio code

So the use case is this - we have a kubernetes cluster in a privately hosted cloud environment and we have the registry proxy deployed in the cluster as an image repository.

The upstream that we use is a private instance of harbor.

The DNS used by the k8s cluster does not have an entry for harbor (and it is not something we can configure without losing vendor support).

For connectivity to the instance of harbor, we have static routes enabling our k8s workers to reach the instance. and by adding an entry to our hosts file we can reach the instance via fqdn.

So in order to complete the solution for our use case, the proxy has to be able to reach the harbor similar to how a hosts file resolution takes place

[rpardini]

Thanks for the explanation. It is indeed a very specific use case. How do you handle TLS validation in this case?

[hishamanver]

So the upstream (harbor) has a certificate that is trusted by our k8s workers <- harbor currently is using Let's Encrypt

But yes, we would have to have trusted certs on the workers in order for this to work, else we would have to use the toggle ssl verification using the provided ENV var in this project

[rpardini]

Sorry, I meant MITM/TLS between Docker/containerd and d-r-p. Just wondering how you can have powers to inject a root CA but not to edit nameservers...

[hishamanver]

I see what you mean. So the platform does not prohibit us from actually changing the nameservers in the code dns config. It's more a case of what the vendor will support with regards to tweaks we make on the cluster. Given that the coredns application directly impacts traffic facilitating services, tweaking this config is not supported.

Also just as an fyi we add the ca for docker by adding it to /etc/docker/certs.d on the k8s workers <- prevents the need for docker restarts (after the first instantiation, which is needed to update the docker proxy file)

[rpardini]

Ahn. So you're running d-r-p as a Pod in a k8s cluster whose nodes' Docker point to it?

[hishamanver]

exactly right :)