swift-server
Fix GPG check randomly fails
Problem
The install script randomly fails with an error like below.
Importing Swift's PGP keys...
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
Cause
Sometimes https://www.swift.org/keys/all-keys.asc returns a gzipped response, regardless of the request header's Accept-Encoding.
Pattern1
< HTTP/1.1 200 OK
< Server: Apple
< Date: Thu, 29 Aug 2024 02:19:07 GMT
< Content-Type: text/plain; charset=UTF-8
< Content-Length: 18563
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Last-Modified: Wed, 28 Aug 2024 07:22:09 GMT
< Accept-Ranges: bytes
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< Cache-Control: max-age=180, public
< Etag: "4883-620b9374b8a40"
< Age: 2
< Via: https/1.1 jptyo7-edge-mx-017.ts.apple.com (acdn/255.14450), https/1.1 jptyo7-edge-fx-003.ts.apple.com (acdn/255.14450)
< X-Cache: hit-stale, hit-stale
< CDNUUID: 5da39fb1-3e44-440b-ab5d-f9dbba698545-515598867
< Connection: keep-alive
<
[plain text output]
Pattern2(gzipped)
< HTTP/1.1 200 OK
< Server: Apple
< Date: Thu, 29 Aug 2024 02:18:41 GMT
< Content-Type: text/plain; charset=UTF-8
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Last-Modified: Wed, 28 Aug 2024 07:22:09 GMT
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< Content-Encoding: gzip
< Cache-Control: max-age=180, public
< Etag: "4883-620b9374b8a40"
< Accept-Ranges: bytes
< Age: 0
< Content-Length: 11040
< Via: https/1.1 jptyo5-edge-lx-012.ts.apple.com (acdn/255.14450), https/1.1 jptyo5-edge-bx-029.ts.apple.com (acdn/255.14450)
< X-Cache: hit-stale, miss
< CDNUUID: 0bcda0aa-252d-45af-a452-36166b9e6d17-382020025
< Connection: keep-alive
<
[binary output]
How to fix
Simply add the --compressed options to the curl command.
This option allows curl to handle gzipped bodies transparently.
Reference
- https://github.com/swift-actions/setup-swift/pull/680
I saw this error in a CI job I'm setting up and noticed that I can't work around it by importing the keys manually and using the --no-import-pgp-keys option, because it's not in the long opts list :(
https://github.com/swiftlang/swiftly/blob/a7ef9e838b21b7f2836f318e67b4bffa6bee259f/install/swiftly-install.sh#L352
This should be fixed in the new swiftly that uses Swift NIO for the http transfer, more robust to 302 redirects.
This should be fixed with the new swiftly 1.0.0 release due to the use of Swift NIO.
https://github.com/swiftlang/swiftly/pull/162#issuecomment-2762535359 Is it right?
It seems swiftly uses .shared (with no proxy) HTTPClient and it will not decompress gzipped response (default is .disabled).
https://github.com/swift-server/async-http-client/blob/01908f4f53751f24ee286353d4489efa125d2070/Sources/AsyncHTTPClient/HTTPClient.swift#L857
@sidepelican iirc the problem was resolved by changing the hostname for swift.org to www.swift.org, which yielded a payload that isn't gzip compressed, or at least one that the shared HTTPClient can process.
Have you encountered any problems with Linux and gpg verifications since the swiftly 1.0.0 release?
My local checking with curl, it seems that www.swift.org is still returning strange responses. However, swiftly 1.0.0 seems working fine. Since the number of attempts is small, it might just be a coincidence.
In any case, since the implementation in this PR is outdated, I’ll close this.
Have you encountered any problems with Linux and gpg verifications since the swiftly 1.0.0 release?
~~Yes. So I had to execute curl --silent --retry 3 --location --fail --compressed https://swift.org/keys/all-keys.asc | gpg --import - before swiftly install.~~
~~The same issue has been fixed in swift-docker. Is there any chance to reopen (re-create?) this PR?~~
~~https://github.com/swiftlang/swift-docker/pull/495~~
I may have made a mistake, so I will double-check.
