adoptium-support icon indicating copy to clipboard operation
adoptium-support copied to clipboard

Getting 403 error randomly when installing temurin packages

Open angaaruriakhil opened this issue 1 year ago • 51 comments

We are conducting ubuntu 20.04 image builds using code in the runner-images repository.

When installing temurin JDK packages, we now get 403 errors randomly on differing packages.

We conducted a build yesterday which failed on installing temurin 21 with the following error:

azure-arm: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.0.0.0+35_amd64.deb 403 Forbidden [IP: 146.75.107.42 443]

We conducted a build today which failed on temurin 11 with the following error:

azure-arm.build_vhd: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-11/temurin-11-jdk_11.0.20.1.0%2b1_amd64.deb Error reading from server. Remote end closed connection [IP: 146.75.107.42 443]

We have also had some successful builds where all the packages install.

To Reproduce

Run code in java-tools.sh to install temurin packages.

Expected behavior Every package is able to be installed reliably

angaaruriakhil avatar Oct 20 '23 14:10 angaaruriakhil

@angaaruriakhil Can you try updating you ca-certificates package and try again?

karianna avatar Oct 20 '23 20:10 karianna

Got the same issue here - even after sudo apt-get install ca-certificates; sudo update-ca-certificates

It worked after an sudo apt update --fix-missing

KrustyDaClown avatar Oct 23 '23 13:10 KrustyDaClown

@gdams over to you - I can only imagine a CDN issue?

karianna avatar Oct 24 '23 03:10 karianna

We are intermittently seeing this issue as well, specifically via yum but I was able to reproduce (albeit with somewhat different error messaging) with curl as well (output below).

Example yum install failure:

$ sudo yum install -y temurin-11-jdk
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd

# [ dependency resolution omitted for brevity's sake ]

Total size: 160 M
Total download size: 159 M
Installed size: 314 M
Downloading packages:
No Presto metadata available for Adoptium
temurin-11-jdk-11.0.20.1.0.1-1 FAILED                                          
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden                                                                                                 ]  0.0 B/s |    0 B  --:--:-- ETA 
Trying other mirror.


Error downloading packages:
  temurin-11-jdk-11.0.20.1.0.1-1.x86_64: [Errno 256] No more mirrors to try.

I encountered this failure several times over starting at 14:55 UTC today, but in my case only for the temurin-11-jdk package; I was able to successfully download and install temurin-11-jre as well as temurin-17-jdk, but temurin-11-jdk continued to fail. As had been suggested on this issue and issue #554, I tried both updating the ca-certificates package and running update-ca-trust – neither of which updated anything – but installation of temurin-11-jdk continued to fail. At this point I tried to curl the failing URL, and I did not receive a 403, but I also wasn't able to download the RPM:

$ curl -vLO https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 146.75.39.42:443...
* Connected to packages.adoptium.net (146.75.39.42) port 443
* ALPN: curl offers h2,http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
{ [5 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [100 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3978 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=packages.adoptium.net
*  start date: Oct  2 11:22:18 2023 GMT
*  expire date: Dec 31 11:22:17 2023 GMT
*  subjectAltName: host "packages.adoptium.net" matched cert's "packages.adoptium.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
} [5 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: packages.adoptium.net]
* [HTTP/2] [1] [:path: /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm]
* [HTTP/2] [1] [user-agent: curl/8.3.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm HTTP/2
> Host: packages.adoptium.net
> User-Agent: curl/8.3.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 200 
< x-amz-id-2: H5ntxCxOdyFhTEZa+JrnjbA2V9VdbpXoFdCeekzjgSGgWSBi/YUbTChNj90JUOuqUxyWHrANR+A=
< x-amz-request-id: QJZ3FQGCMJW8Z84N
< x-amz-replication-status: FAILED
< last-modified: Thu, 31 Aug 2023 12:51:43 GMT
< etag: "71b38a94dbeff6f692cf13a3b7038f35"
< x-amz-server-side-encryption: AES256
< x-amz-version-id: umzWdTNJmMDdvnR0unSw_fRTy4j1NfZd
< content-disposition: attachment;filename="temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm"
< content-type: application/x-rpm
< server: AmazonS3
< cache-control: public, max-age=15780096.000
< fastly-restarts: 1
< accept-ranges: bytes
< date: Tue, 24 Oct 2023 15:13:19 GMT
< via: 1.1 varnish
< age: 3022
< x-served-by: cache-iad-kcgs7200148-IAD
< x-cache: HIT
< x-cache-hits: 1
< x-timer: S1698160399.397782,VS0,VE2
< strict-transport-security: max-age=31557600
< content-length: 166352397
< 
{ [5 bytes data]
* HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
 60  158M   60 96.0M    0     0   166M      0 --:--:-- --:--:-- --:--:--  166M
* Connection #0 to host packages.adoptium.net left intact
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

This failed command actually did download 96M – other attempts downloaded different amounts – but it always failed. The error message implying that the stream wasn't closed cleanly leads me to wonder if the cached asset is corrupt, or perhaps isn't fully-distributed/cached by the CDN? I'm not sure if/how yum is interpreting this as a 403, or if this error is somehow unique/different... meanwhile, downloading the RPM for temurin-17-jdk via curl worked just fine:

$ curl -vLO https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 146.75.35.42:443...
* Connected to packages.adoptium.net (146.75.35.42) port 443
* ALPN: curl offers h2,http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
{ [5 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [100 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [3978 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=packages.adoptium.net
*  start date: Oct  2 11:22:18 2023 GMT
*  expire date: Dec 31 11:22:17 2023 GMT
*  subjectAltName: host "packages.adoptium.net" matched cert's "packages.adoptium.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
} [5 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: packages.adoptium.net]
* [HTTP/2] [1] [:path: /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm]
* [HTTP/2] [1] [user-agent: curl/8.3.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm HTTP/2
> Host: packages.adoptium.net
> User-Agent: curl/8.3.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 200 
< x-amz-id-2: xUHOk3FxG3CALkYV/8zV6BjfkJl7o9PVSMO6jAXu1BBEKzEi8qRBbhPN+6SrCDSncmHJE9JYhqc=
< x-amz-request-id: 9FDX85EN3B8YFEC4
< x-amz-replication-status: FAILED
< last-modified: Thu, 31 Aug 2023 14:32:03 GMT
< etag: "a8ed74787d123dc1a9664b59a4c14ca2"
< x-amz-server-side-encryption: AES256
< x-amz-version-id: Z1zAzlEUB_ubAmeOULSp6EX2L.7XXn9A
< content-disposition: attachment;filename="temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm"
< content-type: application/x-rpm
< server: AmazonS3
< cache-control: public, max-age=15780096.000
< fastly-restarts: 1
< accept-ranges: bytes
< date: Tue, 24 Oct 2023 16:12:30 GMT
< via: 1.1 varnish
< age: 0
< x-served-by: cache-iad-kjyo7100121-IAD
< x-cache: MISS
< x-cache-hits: 0
< x-timer: S1698163950.013750,VS0,VE913
< strict-transport-security: max-age=31557600
< content-length: 162689473
< 
{ [5 bytes data]
100  155M  100  155M    0     0  9496k      0  0:00:16  0:00:16 --:--:-- 10.2M
* Connection #0 to host packages.adoptium.net left intact

Diffing the two downloads, I noted this first download of temurin-17-jdk-17.0.8.1.0.1-1.x86_64.rpm showed a cache MISS; subsequent requests for the same URL were also successful and showed a cache HIT (as did all my requests for temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm), but otherwise they appear functionally similar (no appreciable difference that might point to a reason for the "stream 1 was not closed cleanly" errors that I can see, anyways).

I lost track of the exact time while I was debugging all this, but at some point between 20-30 minutes later, a stray sudo yum install -y temurin-11-jdk completed successfully, and as I can't point to anything I did which resolved this issue on my end, I'm left to assume this issue must be present somewhere on the Artifactory/S3/Fastly-side of things.

jlgoolsbee avatar Oct 24 '23 23:10 jlgoolsbee

Thanks for the responses folks,

I have not seen this issue this week on our builds, we've had 4 successful builds with temurin 8, 11, 17, and 21 installing on each build without issue. Haven't changed anything in our code since my last post. I'm in agreement with the above that it's possibly artifactory/CDN related...

angaaruriakhil avatar Oct 25 '23 02:10 angaaruriakhil

I don't agree that this issue should be closed – I experienced this issue just yesterday, as documented above – and even if we accept the premise that this is a transient issue, it is still a persistent issue that deserves further investigation.

jlgoolsbee avatar Oct 25 '23 15:10 jlgoolsbee

Got the issue again. This time a variable ${VERSION} in the download URL target seems not to be resolved:

olger@phobos:~$ sudo apt upgrade
Paketlisten werden gelesen… Fertig
Abhängigkeitsbaum wird aufgebaut… Fertig
Statusinformationen werden eingelesen… Fertig
Paketaktualisierung (Upgrade) wird berechnet… Fertig
Die folgenden Pakete werden aktualisiert (Upgrade):
  temurin-17-jdk temurin-21-jdk
2 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 343 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 197 kB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n] 
Fehl:1 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 146.75.123.42 443]
Fehl:2 https://packages.adoptium.net/artifactory/deb bookworm/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12
  400  Bad Request [IP: 146.75.123.42 443]
E: Fehlschlag beim Holen von https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb 400  Bad Request [IP: 146.75.123.42 443]
E: Fehlschlag beim Holen von https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb 400  Bad Request [IP: 146.75.123.42 443]
E: Einige Archive konnten nicht heruntergeladen werden; vielleicht »apt-get update« ausführen oder mit »--fix-missing« probieren?

Is says "failed to get" https//packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb https//packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb and I guess the string "${VERSION}" is not intended to be part of the URL...

KrustyDaClown avatar Oct 27 '23 08:10 KrustyDaClown

I'm having the same issue on Linux Mint 20 (Ubuntu 20.04)

The following packages will be upgraded:
  temurin-17-jdk temurin-17-jre
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 195 MB of archives.
After this operation, 2,048 B disk space will be freed.
Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.131.42 443]
Err:2 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-17-jre amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.131.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0+9_amd64.deb  400  Bad Request [IP: 151.101.131.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jre_17.0.9.0+9_amd64.deb  400  Bad Request [IP: 151.101.131.42 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I updated certificates and tried --fix-missing. No joy.

geenweb avatar Oct 27 '23 20:10 geenweb

Same issue here on Ubuntu 22.04 after updating ca-certs and trying --fix-missing

Err:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-17-jdk amd64 17.0.9.0+9
  400  Bad Request [IP: 151.101.3.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-${VERSION}/temurin-17-jdk_17.0.9.0%2b9_amd64.deb  400  Bad Request [IP: 151.101.3.42 443]

dustingrady avatar Oct 27 '23 21:10 dustingrady

Hi folks, we're aware of the issue and are investigating with our CDN and package repo providers.

karianna avatar Oct 27 '23 22:10 karianna

To anyone making an update to a URL that contains -${VERSION} this was an upload error which we've subsequently removed. It appears that artifactory didn't recalculate the index so I've triggered this manually which should fix most user issues. Please can people that were hitting this URL try updating their packages again?

The second bug is around the different behaviour that artifactory serves users if they are in a browser or on a command line. If you make a request in the browser it creates a 301 redirect to a /ui route so that you can explore the repo in artifactory whereas in the command line, it should always serve the binary. I’ve managed to create some code that synthetically handles these 301’s for browser users which should mean that Fastly stops caching the wrong response.

gdams avatar Oct 28 '23 11:10 gdams

Works now. Thank you.

geenweb avatar Oct 28 '23 12:10 geenweb

this is not solved...

wget https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:38:49-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 151.101.23.42 Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.2’

61% [====================================================================> ] 100,663,296 312MB/s in 0.3s

2023-10-31 12:38:50 (312 MB/s) - Connection closed at byte 100663296. Retrying.

--2023-10-31 12:38:51-- (try: 2) https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-10-31 12:38:51 ERROR 403: Forbidden.

wget https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:41:40-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 151.101.23.42 Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.3’

61% [====================================================================> ] 100,663,296 294MB/s in 0.3s

2023-10-31 12:41:40 (294 MB/s) - Connection closed at byte 100663296. Retrying.

--2023-10-31 12:41:41-- (try: 2) https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Connecting to packages.adoptium.net (packages.adoptium.net)|151.101.23.42|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-10-31 12:41:41 ERROR 403: Forbidden. wget --no-cache https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm --2023-10-31 12:43:18-- https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm Resolving packages.adoptium.net (packages.adoptium.net)... 146.75.43.42 Connecting to packages.adoptium.net (packages.adoptium.net)|146.75.43.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 162838233 (155M) [application/x-rpm] Saving to: ‘temurin-17-jdk-17.0.9.0.0.9-1.x86_64.rpm.4’

repudi8or avatar Oct 31 '23 20:10 repudi8or

Hi, we have been experiencing this issue again today.

angaaruriakhil avatar Nov 02 '23 10:11 angaaruriakhil

I'm talking with Fastly again today, I think that Fastly is caching incomplete requests and then continuing to serve them rather than routing users directly to Artificatory. I'll provide an update here once I have one.

gdams avatar Nov 02 '23 11:11 gdams

Still having this issue. Failed to fetch temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb.

Error reading from server. Remote end closed connection

xsnrg avatar Nov 02 '23 20:11 xsnrg

Still having this issue. Failed to fetch temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb.

Error reading from server. Remote end closed connection

@xsnrg can you share the full URL that you are trying to request?

gdams avatar Nov 03 '23 10:11 gdams

sure thing. It seems to be working again this morning.

E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb  Error reading from server. Remote end closed connection [IP: 146.75.39.42 443]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

xsnrg avatar Nov 03 '23 13:11 xsnrg

@gdams: Seems like this issue is happening somewhat frequently for me as well:

22:15:08  [1;31m==> x86_64-ami-builder: https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.20.1.0.1-1.x86_64.rpm: [Errno 14] curl#92 - "HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)"[0m
22:15:08  [1;31m==> x86_64-ami-builder: Trying other mirror.[0m
22:15:08  [1;31m==> x86_64-ami-builder:[0m
22:15:08  [1;31m==> x86_64-ami-builder:[0m
22:15:08  [1;31m==> x86_64-ami-builder: Error downloading packages:[0m
22:15:08  [1;31m==> x86_64-ami-builder:   temurin-11-jdk-11.0.20.1.0.1-1.x86_64: [Errno 256] No more mirrors to try.[0m
22:15:08  [1;31m==> x86_64-ami-builder:[0m

ryantse avatar Nov 03 '23 20:11 ryantse

This is inconsistent and still requires attention.

Most of the time HTTPS Error 403 - Forbidden:

temurin-11-jdk-11.0.21.0.0.9-1 FAILED
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden                      ]  0.0 B/s |    0 B  --:--:-- ETA
Trying other mirror.

In the very rare case the binary is able to download it often fails with HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)

temurin-11-jdk-11.0.21.0.0.9-1 FAILED                                                       30% [=========================                                                         ]  0.0 B/s |  50 MB  --:--:-- ETA
https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] curl#92 - "HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)"-:-- ETA
Trying other mirror.

When persistent, there will a rare occasion when it works (which some people posting above when it "appears to be working again".... are likely getting the lucky one off.

Total download size: 161 M
Installed size: 315 M
Downloading packages:
Public key for temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total                                              187 MB/s | 161 MB  00:00
Retrieving key from https://packages.adoptium.net/artifactory/api/gpg/key/public
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libXau-1.0.8-2.1.amzn2.0.2.x86_64                           1/10
  Installing : libxcb-1.12-1.amzn2.0.2.x86_64                              2/10
  Installing : alsa-lib-1.1.4.1-2.amzn2.x86_64                             3/10
  Installing : libX11-common-1.6.7-3.amzn2.0.4.noarch                      4/10
  Installing : libX11-1.6.7-3.amzn2.0.4.x86_64                             5/10
  Installing : libXext-1.3.3-3.amzn2.0.2.x86_64                            6/10
  Installing : libXi-1.7.9-1.amzn2.0.2.x86_64                              7/10
  Installing : libXtst-1.2.3-1.amzn2.0.2.x86_64                            8/10
  Installing : libXrender-0.9.10-1.amzn2.0.2.x86_64                        9/10
  Installing : temurin-11-jdk-11.0.21.0.0.9-1.x86_64                      10/10
  Verifying  : libXext-1.3.3-3.amzn2.0.2.x86_64                            1/10
  Verifying  : libXrender-0.9.10-1.amzn2.0.2.x86_64                        2/10
  Verifying  : libX11-common-1.6.7-3.amzn2.0.4.noarch                      3/10
  Verifying  : libX11-1.6.7-3.amzn2.0.4.x86_64                             4/10
  Verifying  : libXtst-1.2.3-1.amzn2.0.2.x86_64                            5/10
  Verifying  : alsa-lib-1.1.4.1-2.amzn2.x86_64                             6/10
  Verifying  : libXau-1.0.8-2.1.amzn2.0.2.x86_64                           7/10
  Verifying  : temurin-11-jdk-11.0.21.0.0.9-1.x86_64                       8/10
  Verifying  : libXi-1.7.9-1.amzn2.0.2.x86_64                              9/10
  Verifying  : libxcb-1.12-1.amzn2.0.2.x86_64                             10/10

Installed:
  temurin-11-jdk.x86_64 0:11.0.21.0.0.9-1

inodes avatar Nov 15 '23 05:11 inodes

I am seeing this as well today when installing Adoptium JDK 11 via Flintrock, which installs JDK on instances launched on EC2:

https://packages.adoptium.net/artifactory/rpm/amazonlinux/2/x86_64/Packages/temurin-11-jdk-11.0.21.0.0.9-1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden
Trying other mirror.


Error downloading packages:
  temurin-11-jdk-11.0.21.0.0.9-1.x86_64: [Errno 256] No more mirrors to try.

Is there some more information I can share to help identify the root cause here?

nchammas avatar Nov 20 '23 19:11 nchammas

I was getting 403 Forbidden yesterday, some minutes later it worked ok without doing anything different. This is a very unreliable and flaky networking issue.

I got the chance to get some information in case it is useful to find the root cause by using Ubuntu APT debugging options (-oDebug::Acquire::https=1, for example apt-get install -oDebug::Acquire::https=1 temurin-17-jdk=17.0.9.0.0+9)

Also, I observed that the 403 is not thrown at the very beginning but after some content is already downloaded (~15 %). I guess that APT opens multiple connections to download packages and a one of those workers got the 403 failing the rest of the download:

root@3d636ca4e8cf:/home/user# ls -la /var/cache/apt/archives/partial/
total 16396
drwx------ 1 _apt root     4096 Nov 22 21:36 .
drwxr-xr-x 1 root root     4096 Nov 22 21:29 ..
-rw-r--r-- 1 root root 16777216 Oct 30 14:12 temurin-17-jdk_17.0.9.0.0+9_amd64.deb
root@3d636ca4e8cf:/home/user# rm /var/cache/apt/archives/partial/temurin-17-jdk_17.0.9.0.0+9_amd64.deb

Here are the outputs of APT with debug options when it failed and when it worked:

Failed with 403 downloading latest version
root@3d636ca4e8cf:/home/user# DEBIAN_FRONTEND=noninteractive apt-get install -oDebug::Acquire::https=1 -y --no-install-recommends temurin-17-jdk=17.0.9.0.0+9
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  adoptium-ca-certificates java-common libp11-kit0 p11-kit p11-kit-modules
Suggested packages:
  default-jre
Recommended packages:
  fonts-dejavu-extra
The following NEW packages will be installed:
  adoptium-ca-certificates java-common p11-kit p11-kit-modules temurin-17-jdk
The following packages will be upgraded:
  libp11-kit0
1 upgraded, 5 newly installed, 0 to remove and 68 not upgraded.
Need to get 166 MB/166 MB of archives.
After this operation, 333 MB of additional disk space will be used.
17% [Working]GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1
Host: packages.adoptium.net
User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1)


Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 165543916
x-amz-id-2: 6YwnzVTdwz8yhZkAUYs6NO9vF5PnAQQjCm+5XMRsuNOWikFKSXLJrgAvaI32rRIfzNOALiLxyF8=
x-amz-request-id: YKK0FBYJVA67M8PS
x-amz-replication-status: FAILED
Last-Modified: Mon, 30 Oct 2023 14:12:22 GMT
ETag: "cc468a88d7235d8911184b421b036031"
x-amz-server-side-encryption: AES256
x-amz-version-id: mOcoiHLQ7jeVUun7cgkPTP36Z3b_Hzu8
Content-Type: application/x-debian-package
Server: AmazonS3
Cache-Control: public, max-age=15780096.000
Fastly-Restarts: 1
Accept-Ranges: bytes
Date: Wed, 22 Nov 2023 21:41:36 GMT
Via: 1.1 varnish
Age: 3119
X-Served-By: cache-eze2230026-EZE
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1700689297.581823,VS0,VE12

Get:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9 [166 MB]
24% [1 temurin-17-jdk 13.8 MB/166 MB 8%]GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1
Host: packages.adoptium.net
Range: bytes=16777216-
If-Range: Mon, 30 Oct 2023 14:12:22 GMT
User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1)


25% [1 temurin-17-jdk 16.8 MB/166 MB 10%]Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb
HTTP/1.1 403 Forbidden
Connection: keep-alive
x-amz-request-id: F5S03N8J3B9NJN7T
x-amz-id-2: 9R/nDMoJQWyrBQY14fvoph09ODiZOXaWEtVzKtvsngVZrg0FQIIy06hLuGnHn7NXoxnp88rwOyc=
Content-Type: application/xml
Server: AmazonS3
Fastly-Restarts: 1
Accept-Ranges: bytes
Date: Wed, 22 Nov 2023 21:41:38 GMT
Via: 1.1 varnish
X-Served-By: cache-eze2230058-EZE
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1700689298.102621,VS0,VE749
transfer-encoding: chunked

Err:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9
  403  Forbidden [IP: 151.101.219.42 443]
E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb  403  Forbidden [IP: 151.101.219.42 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
root@3d636ca4e8cf:/home/user#
Downloaded latest version some minutes later
root@3d636ca4e8cf:/home/user# DEBIAN_FRONTEND=noninteractive apt-get install -oDebug::Acquire::https=1 -y --no-install-recommends temurin-17-jdk=17.0.9.0.0+9
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  adoptium-ca-certificates java-common p11-kit p11-kit-modules
Suggested packages:
  default-jre
Recommended packages:
  fonts-dejavu-extra
The following NEW packages will be installed:
  adoptium-ca-certificates java-common p11-kit p11-kit-modules temurin-17-jdk
0 upgraded, 5 newly installed, 0 to remove and 68 not upgraded.
Need to get 166 MB of archives.
After this operation, 332 MB of additional disk space will be used.
0% [Working]GET /artifactory/deb/pool/main/a/adoptium-ca-certificates/adoptium-ca-certificates_1.0.2-1_all.deb HTTP/1.1
Host: packages.adoptium.net
User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1)


Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/a/adoptium-ca-certificates/adoptium-ca-certificates_1.0.2-1_all.deb
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 2304
Content-Type: application/x-debian-package
X-JFrog-Version: Artifactory/7.71.1 77101900
X-Artifactory-Id: 5da408aece77f41502230d8ecf66e33487a6aab2
X-Artifactory-Node-Id: adoptium-artifactory-primary-0
Last-Modified: Tue, 20 Jun 2023 16:04:14 GMT
ETag: 0347ba993198898a773c8ba4084e8022d3dd1bfb
X-Checksum-Sha1: 0347ba993198898a773c8ba4084e8022d3dd1bfb
X-Checksum-Sha256: f944bcc23005177751212e578437065d6eedbfa57e26b1821b7f12456dcb5413
X-Checksum-Md5: 5db8bf5481fe4ad127e500d9ffc4576f
X-Artifactory-Filename: adoptium-ca-certificates_1.0.2-1_all.deb
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Request-ID: 3554394ae9c346a390b48a1de9713501
Cache-Control: public, max-age=15780096.000
Accept-Ranges: bytes
Date: Wed, 22 Nov 2023 21:50:36 GMT
Via: 1.1 varnish
Age: 1226858
X-Served-By: cache-eze2230064-EZE
X-Cache: HIT
X-Cache-Hits: 2
X-Timer: S1700689837.703204,VS0,VE0

Get:1 https://packages.adoptium.net/artifactory/deb bionic/main amd64 adoptium-ca-certificates all 1.0.2-1 [2304 B]
GET /artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0%2b9_amd64.deb HTTP/1.1
Host: packages.adoptium.net
User-Agent: Debian APT-HTTP/1.3 (1.6.12ubuntu0.1)


Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 p11-kit-modules amd64 0.23.9-2ubuntu0.1 [234 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 p11-kit amd64 0.23.9-2ubuntu0.1 [93.2 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 java-common all 0.68ubuntu1~18.04.1 [14.5 kB]
16% [Waiting for headers]Answer for: https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-17/temurin-17-jdk_17.0.9.0.0+9_amd64.deb
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 165543916
x-amz-id-2: I31eBDk1re2GAh0BEOv+Nd0qnZVHNhMCAtTxW0InaJczvUInPLeqy1r81ucR0TlgaX2exAZDsEc=
x-amz-request-id: CX5K6GGZFPCHNFPD
x-amz-replication-status: FAILED
Last-Modified: Mon, 30 Oct 2023 14:12:22 GMT
ETag: "cc468a88d7235d8911184b421b036031"
x-amz-server-side-encryption: AES256
x-amz-version-id: mOcoiHLQ7jeVUun7cgkPTP36Z3b_Hzu8
Content-Type: application/x-debian-package
Server: AmazonS3
Cache-Control: public, max-age=15780096.000
Fastly-Restarts: 1
Accept-Ranges: bytes
Date: Wed, 22 Nov 2023 21:50:40 GMT
Via: 1.1 varnish
Age: 2
X-Served-By: cache-eze2230064-EZE
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1700689837.713358,VS0,VE3556

Get:5 https://packages.adoptium.net/artifactory/deb bionic/main amd64 temurin-17-jdk amd64 17.0.9.0.0+9 [166 MB]
Fetched 166 MB in 49s (3365 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package p11-kit-modules:amd64.
(Reading database ... 25417 files and directories currently installed.)
Preparing to unpack .../p11-kit-modules_0.23.9-2ubuntu0.1_amd64.deb ...
Unpacking p11-kit-modules:amd64 (0.23.9-2ubuntu0.1) ...
Selecting previously unselected package p11-kit.
Preparing to unpack .../p11-kit_0.23.9-2ubuntu0.1_amd64.deb ...
Unpacking p11-kit (0.23.9-2ubuntu0.1) ...
Selecting previously unselected package adoptium-ca-certificates.
Preparing to unpack .../adoptium-ca-certificates_1.0.2-1_all.deb ...
Unpacking adoptium-ca-certificates (1.0.2-1) ...
Selecting previously unselected package java-common.
Preparing to unpack .../java-common_0.68ubuntu1~18.04.1_all.deb ...
Unpacking java-common (0.68ubuntu1~18.04.1) ...
Selecting previously unselected package temurin-17-jdk.
Preparing to unpack .../temurin-17-jdk_17.0.9.0.0+9_amd64.deb ...
Unpacking temurin-17-jdk (17.0.9.0.0+9) ...
Setting up java-common (0.68ubuntu1~18.04.1) ...
Setting up p11-kit-modules:amd64 (0.23.9-2ubuntu0.1) ...
Setting up p11-kit (0.23.9-2ubuntu0.1) ...
Setting up adoptium-ca-certificates (1.0.2-1) ...
/etc/ssl/certs/adoptium/cacerts successfully populated.
Setting up temurin-17-jdk (17.0.9.0.0+9) ...
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jar to provide /usr/bin/jar (jar) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javac to provide /usr/bin/javac (javac) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/javap to provide /usr/bin/javap (javap) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdeprscan to provide /usr/bin/jdeprscan (jdeprscan) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jfr to provide /usr/bin/jfr (jfr) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jhsdb to provide /usr/bin/jhsdb (jhsdb) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jimage to provide /usr/bin/jimage (jimage) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jlink to provide /usr/bin/jlink (jlink) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jmod to provide /usr/bin/jmod (jmod) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jpackage to provide /usr/bin/jpackage (jpackage) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jps to provide /usr/bin/jps (jps) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jshell to provide /usr/bin/jshell (jshell) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode
update-alternatives: using /usr/lib/jvm/temurin-17-jdk-amd64/lib/jspawnhelper to provide /usr/bin/jspawnhelper (jspawnhelper) in auto mode
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
root@3d636ca4e8cf:/home/user#

I suggest looking to x-amz-replication-status (AWS doc, and how it can be monitored), the X-Cache and Age headers.

Just for reference, these are other reported issues that I come across while researching the problem.... it seems that the issue is not new:

  • https://github.com/adoptium/adoptium-support/issues/932
  • https://github.com/adoptium/adoptium-support/issues/632
  • https://github.com/adoptium/adoptium-support/issues/554

mdaloia avatar Nov 23 '23 13:11 mdaloia

We too are facing this issue everyday, and it is not clear why.... Only during some moments we can actually install the package... is there any alternative for https://packages.adoptium.net/artifactory/deb that it is more reliable? or any known mitigation for this?

Thank you

saamorim avatar Dec 14 '23 18:12 saamorim

I can confirm that we are as well seeing this intermittent issue. Certs do not appear to be the issue. Bumping this thread. 👍

Konig-Corey avatar Dec 26 '23 18:12 Konig-Corey

Seeing similar errors with debian package manager (apt) as well several times per day but not always.

hpernu avatar Jan 11 '24 11:01 hpernu

Seem to be getting this error here still:

azure-arm.build_image: The following NEW packages will be installed: azure-arm.build_image: temurin-21-jdk azure-arm.build_image: 0 upgraded, 1 newly installed, 0 to remove and 19 not upgraded. azure-arm.build_image: Need to get 177 MB of archives. azure-arm.build_image: After this operation, 360 MB of additional disk space will be used. azure-arm.build_image: Err:1 https://packages.adoptium.net/artifactory/deb jammy/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 azure-arm.build_image: 403 Forbidden [IP: 146.75.39.42 443] ==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0%2b12_amd64.deb 403 Forbidden [IP: 146.75.39.42 443] ==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ==> azure-arm.build_image: cat: /usr/lib/jvm/temurin-21-jdk-amd64/release: No such file or directory azure-arm.build_image: Java 21 Toolcache Version Path: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk//usr/lib/jvm/temurin-21-jdk-amd64/bin/java: azure-arm.build_image: Setting up JAVA_HOME_21_X64 variable to /usr/lib/jvm/temurin-21-jdk-amd64 azure-arm.build_image: JAVA_HOME_21_X64=/usr/lib/jvm/temurin-21-jdk-amd64

The JAVA_HOME_21_X64 is being set and thus this fails:

azure-arm.build_image: [-] Java 21 33ms (30ms|3ms) azure-arm.build_image: Command '"/usr/lib/jvm/temurin-21-jdk-amd64/bin/java" -version' has finished with exit code azure-arm.build_image: /usr/bin/bash: line 1: /usr/lib/jvm/temurin-21-jdk-amd64/bin/java: No such file or directory azure-arm.build_image: at ""$javaPath" -version" | Should -ReturnZeroExitCode, /imagegeneration/tests/Java.Tests.ps1:43 azure-arm.build_image: at <ScriptBlock>, /imagegeneration/tests/Java.Tests.ps1:43

I was having other issues on my Windows build of my runner-images build, which I attributed to Defender. Been adding the 'ExcludeMdeAutoProvisioning' tag to disable provisioning on both the Windows and Linux builds, but this seems to not be an issue on the Azure side, but the adoptium endpoint. I had a similar issue with VS extensions and the UI behavior doing a redirect that CLI was not getting properly, which resulted in me adding logic that sets redirect to "0" then gets the resulting 'Location' header to make the call correctly. I was then getting a CDN error due to limits on calls to the service in a given time. This came back with a 'Retry-After' header, that I used to retry the call after the time I was given. This worked for me. Not sure if something similar is needed here. This will be dependent on the CDN giving a response with a header that can be used.

DevOpsAzurance avatar Jan 18 '24 19:01 DevOpsAzurance

Hey folks, once we've worked through the latest PSU release we'll be getting back to investigating this. Just wanted to drop a note that this isn't being ignored :-)

karianna avatar Jan 22 '24 08:01 karianna

Hi @karianna I am facing an issue with devops builds and was notified that this issue is blocking me.

https://github.com/actions/runner-images/issues/9220 this is the issue i am facing, Let us know if there is any workaround.

azure-arm.build_image: Get:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 [177 MB] azure-arm.build_image: Err:1 https://packages.adoptium.net/artifactory/deb focal/main amd64 temurin-21-jdk amd64 21.0.1.0.0+12 azure-arm.build_image: 403 Forbidden [IP: 146.75.107.42 443] ==> azure-arm.build_image: E: Failed to fetch https://packages.adoptium.net/artifactory/deb/pool/main/t/temurin-21/temurin-21-jdk_21.0.1.0.0+12_amd64.deb 403 Forbidden [IP: 146.75.107.42 443] ==> azure-arm.build_image: E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

vamshisiram avatar Jan 24 '24 15:01 vamshisiram

It likely is, once we've completed this PSU release cycle folks will take a fresh look.

karianna avatar Jan 24 '24 19:01 karianna

@karianna when is that release cycle? Just so i can check back and keep informing my company internally on when we can expect stuff to work again.

vamshisiram avatar Jan 24 '24 23:01 vamshisiram