vector mirror for docker images

docker hub has pull limits https://www.docker.com/increase-rate-limits Please create docker image mirrors on alternative sites. For example https://quay.io/ thank

Mar 11 '21 10:03 ivmaks

or https://github.com/features/packages

Mar 12 '21 10:03 ivmaks

Any chance this will happen anytime soon?

Jun 18 '21 15:06 alvaroaleman

~It seems like it'd be pretty easy for us to mirror to Github's registry, https://ghcr.io. Would that work for you @alvaroaleman @ivmaks ?~ Actually scratch that, I'm not sure we could use that registry. I'll dig around a bit more.

Jun 25 '21 16:06 jszwedko

I think we could publish to gcr.io and/or public.ecr.aws.

Jun 25 '21 16:06 jszwedko

Any registry that allows unauthenticated pulls without ratelimiting would work

Jun 25 '21 16:06 alvaroaleman

Hello! Any progress?

Aug 26 '21 12:08 ykizin

Hi @ykizin ! Not yet, but it is on our radar.

Sep 15 '21 19:09 jszwedko

Any news on when this can be available?

Mar 21 '22 10:03 mbastawisy

I expect this issue to be closed by https://github.com/vectordotdev/vector/issues/10969 - which is currently on our roadmap to start 4/4/2022

Mar 21 '22 13:03 spencergilbert

Any update on this? Dockerhub really isn't OSS friendly anymore. They are pushing to enterprise and paid customers.

GHCR seems like an easy win.

Dec 18 '22 01:12 onedr0p

Hey @onedr0p - unfortunately it hasn't been a high priority for us recently. The likely plan is to co-opt Datadog's existing infrastructure here and publish to gcr.io/datadoghq, docker.io/datadog, public.ecr.aws/datadog - but we've ended up needing to shift other work in place of that.

That's on our list again this quarter - hopefully community need can help us keep it on the TODO list. cc @jszwedko for visibility.

Dec 19 '22 13:12 spencergilbert

Since you asked... I might as well add another +1 comment.

Publishing it to public.ecr.aws would be much appreciated as that's our main cloud provider and all Vector usage are in there for us currently. 🙏

Jan 19 '23 14:01 aquarhead

It would be great to bump the priority on this, no one wants to have their observability into logs stopped because of dockerhubs silly rate limits. I'm still astonished that Grafana only pushes all their project images to dockerhub as well 😂

I digress but look forward to not having to rely on dockerhub someday <3

Jan 19 '23 15:01 onedr0p

Hey @onedr0p - unfortunately it hasn't been a high priority for us recently. The likely plan is to co-opt Datadog's existing infrastructure here and publish to gcr.io/datadoghq, docker.io/datadog, public.ecr.aws/datadog - but we've ended up needing to shift other work in place of that.

That's on our list again this quarter - hopefully community need can help us keep it on the TODO list. cc @jszwedko for visibility.

I assume it didn't ship last quarter? Is it on the list for this quarter? We're running into this as well :(

Feb 15 '23 20:02 jalaziz

Given the recent dockerhub announcement of removing free team orgs, I'm wondering if the current timberio org will be affected? And if so perhaps this could finally have a high priority?

Mar 15 '23 12:03 aquarhead

Ref: https://github.com/docker/hub-feedback/issues/2314

Mar 15 '23 17:03 onedr0p

We are using a paid dockerhub account and so won't be affected by this change. We do still aim to publish to other registries in the future though.

Mar 15 '23 17:03 jszwedko

Is there any ETA? Having docker hub as a image source for daemonset is pretty painful nowdays due to their limits

May 25 '23 06:05 dm3ch

I wonder with the effort in the latest release to move linux packages to another registry if could this also be prioritized now?

Nov 07 '23 18:11 onedr0p

This (rather simple to fix) issue is now open for more than 3 years. I want to add my +1 onto this.

Aug 12 '24 11:08 applike-ss

DockerHub has just announced a dramatically tighter rate limit now of 10 pulls per hour per IP address. This issue really needs to be revisited to push official releases to ECR and/or ghcr.io.

For primarily this reason I have had to mirror the official vector releases into our company ECR for the last couple years since we started using vector because it is such a critical DaemonSet on every one of our EKS nodes that if rate limiting hits it can significantly impact operations.

Thanks!

Feb 21 '25 17:02 mmerickel

:+1:

This is especially critical when k8s nodes sit behind a NAT gateway with a single IP address and nodes get rotated out.

Feb 21 '25 17:02 archoversight

Thank you for your comments on this. We will take a look this week.

Feb 24 '25 17:02 pront

@pront any update on this issue? Dockerhub pushed back the enforcement of the new rate limit to April 1st but the date is getting closer.

Mar 17 '25 17:03 danielricoh

@pront any update on this issue? Dockerhub pushed back the enforcement of the new rate limit to April 1st but the date is getting closer.

Hi @danielricoh, this is still on my TODO list. I will update this ticket whenever progress is made.

Mar 17 '25 17:03 pront

Custom builds are now published in https://github.com/vectordotdev/vector/pkgs/container/vector. Keeping this open until the next release, it will be the first release that will attempt to also publish the Docker images to GHCR.

Mar 21 '25 19:03 pront

https://github.com/vectordotdev/vector/pkgs/container/vector/versions?filters%5Bversion_type%5D=tagged&page=1

Apr 11 '25 14:04 pront