testbed icon indicating copy to clipboard operation
testbed copied to clipboard

Issues Downloading Docker Images During Playbook Execution

Open Richie1710 opened this issue 1 year ago • 2 comments

Hello, I've encountered an issue while running the playbook where certain Docker images could not be downloaded from the Nexus repository. The playbook attempts to pull the images, but it results in a 500 Internal Server Error. Here are the details:

Image: nexus.testbed.osism.xyz:8193/kolla/keystone-fernet:2023.2
    Error Message: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.45/images/create?tag=2023.2&fromImage=nexus.testbed.osism.xyz%3A8193%2Fkolla%2Fkeystone-fernet: Internal Server Error ("unknown: Remote Auto Blocked until 2024-08-13T07:45:05.681Z")

Image: nexus.testbed.osism.xyz:8193/kolla/haproxy:2023.2
    Error Message: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.45/images/create?tag=2023.2&fromImage=nexus.testbed.osism.xyz%3A8193%2Fkolla%2Fhaproxy: Internal Server Error ("unknown: Remote Auto Blocked until 2024-08-13T07:45:05.681Z")

It seems like the access to the remote repository might be temporarily blocked or there could be an issue with the Nexus server.

Could you please take a look at this? Any assistance would be greatly appreciated.

Thank you!

Richie1710 avatar Aug 13 '24 07:08 Richie1710

Looks like the external repository server cannot be reached. nexus.testbed.osism.xyz:8193 points to our osism.harbor.regio.digital server. It's probably better to use quay.io instead.

https://support.sonatype.com/hc/en-us/articles/213464888-A-proxy-repository-is-not-working-status-says-Remote-Automatically-Blocked-and-Unavailable

If a remote is responding very slowly, this can cause incoming requests to pile up waiting for this one particular remote to respond.  In extreme cases so many requests can pile up that the inbound request thread pool becomes exhausted, rendering the server unusable.

To solve this problem, we added the auto-block feature.  If a remote does not respond within the read timeout setting (default is 20 seconds, with 3 retries) it is auto-blocked. 

berendt avatar Aug 15 '24 17:08 berendt

Seems to be working with images from quay.io

Richie1710 avatar Sep 02 '24 17:09 Richie1710