`ContainerRegistryHook ` can't add image prefix

[afscrome]

Prior to preview 6, I had my own version of ContainerRegistryHook and was hoping to move over to the built in one introduced in preview 6. Unfortunately it doesn't meet our needs so we're having to continue with our custom implementation.

1. Not only do we need to change the registry name, but we also need to add a prefix to the image name. e.g. we want to convert docker.io/redis:latest --> mycompanymirror.azurecr.io/library/docker.io/redis:latest.

2. We only want to rewrite the public image registries, not our own internal ones. (Currently our logic is to rewrite everything that isn't *.azurecr.io, which whilst not 100% true, works well enough without having to maintain an allow list of all our internal ACRs). It would be helpful if there was an allow/deny list of what registries should get overwritten. Examples of when this may be needed

• Your internal mirror and application registries are in separate container registries.
• Different teams in your company push to different registries internally, but you want to rewrite all public images to the same internal mirror without rewriting the internal apps.

Could you update the built in ContainerRegistryHook to account for this? I think you may want to consider this prior to GA as it may impact the API of how this is configured on DistributedApplicationOptions

[davidfowl]

cc @mitchdenny

[mitchdenny]

@afscrome we are all buttoned up for GA. So this won't make that :) Can you give me an example of what you think the API would look like to specify the allow list of registries?

I was thinking something like this:

var builder = DistributedApplication.CreateBuilder(new () {
  AllowedRegistries = ["foo.mycompany.com", "var.mycompany.com" ]
});

[davidfowl]

That setting should be on the hook no?

[mitchdenny]

How would folks set it though? The existing registry setting is on the options.

[afscrome]

In terms of API, I think the lowest level api is to provide a function that takes the registry & image that can then be used to rewrite the registry.

// Simple example
builder.AddContainerRewrite((registry, image) => return $"my.registry/mirrors/{registry}/{image}");

/// More complex scenario - each upstream has it's own repository in the mirrored registry, but the repository doesn't match 1:1 with the upstream registry's domain name.
builder.AddContainerRewrite((registry, image) => registry switch => {
   "docker.io" => $"my.registry.com/docker/{image}"
   "quay.io" => $"my.registry.com/quay/{image}"
   _ => registry;
});


// Only rewrite some registries
builder.AddContainerRewrite((registry, image) => {
  if (registry.EndsWith("azurecr.io") {
    return $"{registry}/{image}"; 
  }
  else {
    return $"my.registry/mirrors/{registry}/{image}"
  }
});

As long as this API is available, you can then build whatever rewrite logic your company needs.

You can then build some more opinionated implementations of how to do registry rewriting on top of that. I'm less sure what those opinionated ways should be - yes my use case is to have an allow list of registries to ignore for rewriting, but I don't know if that's a general / common enough pattern to encode into aspire itself, or whether that's a scenario I should handle through my own delegate passed to AddContainerRewrite.

One possible issue with the above api - it does somewhat go against the rest of aspire which always keeps registry, image and tag separate, rather than encoding them into one string like the above examples.

[mitchdenny]

There is an argument here that doing what you are doing (by registering your own hook) is a solid approach for folks that want to do this. Its trivially easy to implement a hook, do we need to introduce another abstraction?

[davidfowl]

I agree, I'm not sure we need ours 😄

[mitchdenny]

Security ;)

[mitchdenny]

OK I'm going to leave this in backlog for now to see if it collects more feedback. The current recommendation is people use WithRegistry liberally, use our global override, or write their own hook depending on the degree of control and how often they want to repeat themselves.