Proposal: Support to mirror multiple upstream registries

[jeffyjf]

Description

I noticed that distribution currently support to mirror only one upstream registry at a time [1]. But, In many use cases, the container images come from multiple registries (eg: In a kubernetes cluster). So, the feature what support to mirror multiple upstream registries is necessary.

The possible implementation way:

• containerd is the most popular CSI runtime of kubernetes. And it support Registry Host Namespace [2] feature. By this feature, containerd will add a query parameter ?ns=<source registry host> when pulling image with mirror endpoint. So I propose to add several configuration options like blow:

proxy:
   remotehostquerykey: ns
   remotehostconfigmap:
      registry.k8s.io:
          remoteurl: https://registry.k8s.io
          username: username
          password: password

With this, distribution can identify the source registry of the image, obtain the correct remote URL, and pull and cache images from different registries.

[1] https://distribution.github.io/distribution/recipes/mirror/#gotcha [2] https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-host-namespace

[jeffyjf]

@milosgajdos @Jamstah @wy65701436 asking for review. Thank you in advance.

[milosgajdos]

Dupe of https://github.com/distribution/distribution/issues/3725