Add `DockerRegistry` block for `DockerContainer` to pull images from registries with authentication

[zanieb]

• Adds a DockerRegistry block with a login capability.
• Adds an image_registry setting to the DockerContainer infrastructure block allowing login to a registry before image pull.

Addresses https://github.com/PrefectHQ/prefect/issues/6230 — I do not think it is fair to close this since a separate implementation with ECR token refresh support will be required.

[zanieb]

I'm not 100% sure on ECR, you can see some examples in the upstream issue https://github.com/docker/docker-py/issues/2256#issuecomment-887175827

[anna-geller]

This is awesome, thanks so much Michael!

@chrisguidry yes correct, you would use this command to get the token which can be used as password, the username is always AWS:docker login -u AWS -p your_token

In theory, AWS considers this as insecure and recommends using --password-stdin:

aws ecr get-login-password --region region | docker login --username AWS --password-stdin aws_account_id.dkr.ecr.region.amazonaws.com

but either way, the token is only valid for 12 hours, so not sure whether after this initial login, dockerpy can consider just using permissions attached via IAM role to the EC2 instance...

[zanieb]

We can add an ElasticContainerRegistry block that pulls the ECR token at runtime to the prefect-aws collection. cc @desertaxle

[anna-geller]

This works 🎉

QAed with S3 storage block and Dockerfile that does nothing but adding s3fs dependency:

FROM prefecthq/prefect:dev-python3.9
RUN pip install s3fs

[zanieb]

@anna-geller This was waiting for a few things still :) I'll open a follow-up.

[feliperazeek]

@anna-geller I am going over your docker-based sample project, how would I modify it to use ECR? thank you very much!

I know I could define auth through DockerRegistry but how would it handle ECR's auth mechanism where the credentials expire every few hours?

dockerhub = DockerRegistry(
    username="annaprefect",
    password="xxx",
    reauth=True,
    registry_url="https://index.docker.io/v1/",
)

I tried using ElasticContainerRegistry:

registry = ElasticContainerRegistry(registry_id="test")
container = DockerContainer(image_registry=registry)
uuid = container.save("docker", overwrite=True)

Getting an error though with Prefect 2.13.x:

TypeError: Can't instantiate abstract class ElasticContainerRegistry with abstract method get_docker_client

Wondering if you got an example where you got it to work.

Thank you!

[zanieb]

@feliperazeek I'd recommend opening a new issue instead of bumping this old pull request.