pkl
pkl copied to clipboard
apple
`project package` throws `InvalidAlgorithmParameterException` when adding `Package.apiTests`
Given this source code: https://github.com/StefMa/pkl-gha/tree/9d5b220b0a2dcfb971a56b875124a89f6acff611
If you checkout the code and run pkl test tests/*.pkl it runs the tests and they pass.
If you package the code, it works as well ./pkl project package.
As soon as you uncomment line 9 in PklProject (enabling apiTests) and run the package command, the tests pass but afterwards it throws an exception:
InvalidAlgorithmParameterException
module com.github.GitHubAction.tests (file:///~pkl/pkl-github/tests/GitHubAction.pkl, line 1)
basic ✅
An unexpected error has occurred. Would you mind filing a bug report?
org.pkl.core.packages.PackageLoadError: Exception when making request `GET https://github.com/stefma/[email protected]`:
Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at org.pkl.core.packages.PackageResolvers$AbstractPackageResolver.openExternalUri(PackageResolvers.java:211)
at org.pkl.core.packages.PackageResolvers$AbstractPackageResolver.getDependencyMetadataAndComputeChecksum(PackageResolvers.java:103)
at org.pkl.core.project.ProjectPackager.checkAlreadyPublishedPackage(ProjectPackager.java:175)
at org.pkl.core.project.ProjectPackager.doPackage(ProjectPackager.java:162)
at org.pkl.core.project.ProjectPackager.createPackages(ProjectPackager.java:122)
at org.pkl.cli.CliProjectPackager.doRun(CliProjectPackager.kt:88)
at org.pkl.commons.cli.CliCommand.run(CliCommand.kt:45)
at org.pkl.cli.commands.ProjectCommand$Companion$PackageCommand.run(ProjectCommand.kt:145)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:198)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:18)
at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:400)
at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:397)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:415)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:440)
at org.pkl.cli.Main$main$1.invoke(Main.kt:52)
at org.pkl.cli.Main$main$1.invoke(Main.kt:40)
at org.pkl.commons.cli.CliMainKt.cliMain(CliMain.kt:31)
at org.pkl.cli.Main.main(Main.kt:40)
Caused by: javax.net.ssl.SSLException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at [email protected]/sun.security.ssl.Alert.createSSLException(Alert.java:133)
at [email protected]/sun.security.ssl.TransportContext.fatal(TransportContext.java:378)
at [email protected]/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at [email protected]/sun.security.ssl.TransportContext.fatal(TransportContext.java:316)
at [email protected]/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1708)
at [email protected]/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:470)
at [email protected]/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
at [email protected]/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:589)
at [email protected]/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:187)
at [email protected]/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
at [email protected]/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
at [email protected]/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
at [email protected]/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
at org.pkl.core.packages.PackageResolvers$AbstractPackageResolver.openExternalUri(PackageResolvers.java:206)
... 19 more
Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at [email protected]/sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:101)
at [email protected]/sun.security.validator.Validator.getInstance(Validator.java:181)
at [email protected]/sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:309)
at [email protected]/sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:183)
at [email protected]/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:198)
at [email protected]/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
at [email protected]/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1341)
at [email protected]/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
at [email protected]/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
at [email protected]/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
at [email protected]/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480)
at [email protected]/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:458)
at [email protected]/sun.security.ssl.TransportContext.dispatch(TransportContext.java:201)
at [email protected]/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at [email protected]/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
at [email protected]/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
at [email protected]/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
... 27 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at [email protected]/java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
at [email protected]/java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120)
at [email protected]/java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104)
at [email protected]/sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:98)
... 43 more
I have the feeling the baseUri is not correct.
But actually I also don't know what should be a current basiUri.
I also tried package://pkg.pkl-lang.org/github.com/stefma/pkl-gha/\(name) (which sounds to me more correct)
or even set a "valid" version (0.0.1).
Still no success, same error but with a different URL:
org.pkl.core.packages.PackageLoadError: Exception when making request `GET https://pkg.pkl-lang.org/github.com/stefma/pkl-gha/[email protected]`:
All of this sounds strange to me for two reasons:
- Calling this, will successfully download my package (not even sure if this is intended), so why it is an exception?
- Why is there called a URL (with the new version) in this process anyways? In fact, when I do
packageI do a new release, obvious that this is not yet available.
The package command will check if there is already an existing package that exists, but with a different checksum. If it exists, it means that you need to bump your package's version.
You can disable this behavior using the flag --skip-publish-check.
The error that you're seeing suggests that you've set up Pkl with custom CA certificates, but there are problems with your custom certificates.
Do you have a ~/.pkl/cacerts directory on your machine? If so, you will want to ensure that it has all of the root certs that you intend to trust.
Could you check if Pkl built from the main branch gives a better error message that doesn't include "Would you mind filing a bug report?"? I'm asking because my HttpClient PR was recently merged.
@bioball
The package command will check if there is already an existing package that exists, but with a different checksum. If it exists, it means that you need to bump your package's version.
But, why does this only happen when I add the apiTests config? 🤔
Shouldn't it do it all the time?
The error that you're seeing suggests that you've set up Pkl with custom CA certificates
Hmm.. How do i set up such a certificate? If I did so, I never did this on purpose actually.
Do you have a ~/.pkl/cacerts directory on your machine?
Nope, its not there 🙃
@translatenix
Could you check if Pkl built from the main branch
I checked out https://github.com/apple/pkl/tree/8dc258ef7d795809079c0346995ca052fd6aeaed (This includes your changes).
Then I build the cli with ./gradlew :pkl-cli:build and run VERSION=0.0.1 /path/to/checkout/pkl-repo/pkl-cli/build/executable/jpkl project package.
I still get an error, also still including the An unexpected error has occurred. Would you mind filing a bug report? message 😬
org.pkl.core.packages.PackageLoadError:
module com.github.GitHubAction.tests (file:///~pkl/pkl-github/tests/GitHubAction.pkl, line 1)
basic ✅
An unexpected error has occurred. Would you mind filing a bug report?
org.pkl.core.packages.PackageLoadError: Received unexpected status code `301` when making request `GET https://pkg.pkl-lang.org/github.com/stefma/pkl-gha/[email protected]`.
at org.pkl.core.packages.PackageResolvers$AbstractPackageResolver.openExternalUri(PackageResolvers.java:208)
at org.pkl.core.packages.PackageResolvers$AbstractPackageResolver.getDependencyMetadataAndComputeChecksum(PackageResolvers.java:101)
at org.pkl.core.project.ProjectPackager.checkAlreadyPublishedPackage(ProjectPackager.java:177)
at org.pkl.core.project.ProjectPackager.doPackage(ProjectPackager.java:164)
at org.pkl.core.project.ProjectPackager.createPackages(ProjectPackager.java:124)
at org.pkl.cli.CliProjectPackager.doRun(CliProjectPackager.kt:89)
at org.pkl.commons.cli.CliCommand.run(CliCommand.kt:39)
at org.pkl.cli.commands.ProjectCommand$Companion$PackageCommand.run(ProjectCommand.kt:145)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:198)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:211)
at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:18)
at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:400)
at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:397)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:415)
at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:440)
at org.pkl.cli.Main$main$1.invoke(Main.kt:52)
at org.pkl.cli.Main$main$1.invoke(Main.kt:40)
at org.pkl.commons.cli.CliMainKt.cliMain(CliMain.kt:31)
at org.pkl.cli.Main.main(Main.kt:40)
I did a closer look into this and found the following:
In case there is an apiTest property in the PklProject, the ProjectCommand creates the CliProjectPackager.
The CliProjectPackager gets the baseOptions (this is a instance of CliBaseOptions) as arguments.
The baseOptions function will call getEffectiveCaCertificates() to populate the certificates.
In my case, without a custom certificate, the "default" one from the resources directory.
The CliProjectPackager will, in case the package.apiTests property is available, create a new instance of the CliTestRunner.
This Runner gets a copy of the CliBaseOptions.
The options, put to the CliTestRunner, are just a copy of the same options as the CliProjectPackager have.
Thus, it includes the same instance of the caCertificates property❗
Since the base class of both (the CliProjectPackager and the CliTestRunner) is the CliCommand, which has a init function that calls CertificateUtils.setupAllX509CertificatesGlobally followed by CertificateUtils.toInputStream and CertificateUtils.generateCertificates, it will crash at this point because the exact same InputStream is used but this InputStream is already closed.
This is my visual output (println debugging 😁 ):
I call CertificateUtils.setupAllX509CertificatesGlobally from CliProjectPackager
toInputStream! sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@3d299e3
Generating certificates.. Already closed?: 218384
I call CertificateUtils.setupAllX509CertificatesGlobally from CliTestRunner
// Note the same instance! 👇
toInputStream! sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@3d299e3
// At this point, the stream is already closed 👇
Generating certificates.. Already closed?: 0
This is the diff on top of https://github.com/apple/pkl/tree/11f07d1ce8fe3af1e07b3472781f62805b8c419c:
diff --git a/pkl-cli/src/main/kotlin/org/pkl/cli/CliProjectPackager.kt b/pkl-cli/src/main/kotlin/org/pkl/cli/CliProjectPackager.kt
index da000ad..d84e09f 100644
--- a/pkl-cli/src/main/kotlin/org/pkl/cli/CliProjectPackager.kt
+++ b/pkl-cli/src/main/kotlin/org/pkl/cli/CliProjectPackager.kt
@@ -41,6 +41,8 @@ class CliProjectPackager(
val normalizeApiTests = apiTests.map { project.projectDir.resolve(it).toUri() }
val testRunner =
CliTestRunner(
+ // This might be a fix?
+ // cliOptions.copy(sourceModules = normalizeApiTests, projectDir = project.projectDir, caCertificates = emptyList()),
cliOptions.copy(sourceModules = normalizeApiTests, projectDir = project.projectDir),
testOptions = testOptions,
consoleWriter = consoleWriter,
diff --git a/pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliCommand.kt b/pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliCommand.kt
index 948c445..c26c6d8 100644
--- a/pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliCommand.kt
+++ b/pkl-commons-cli/src/main/kotlin/org/pkl/commons/cli/CliCommand.kt
@@ -32,6 +32,7 @@ import org.pkl.core.util.IoUtils
abstract class CliCommand(protected val cliOptions: CliBaseOptions) {
init {
if (cliOptions.caCertificates.isNotEmpty()) {
+ println("I call CertificateUtils.setupAllX509CertificatesGlobally from ${this::class.simpleName}")
CertificateUtils.setupAllX509CertificatesGlobally(cliOptions.caCertificates)
}
}
diff --git a/pkl-core/src/main/java/org/pkl/core/runtime/CertificateUtils.java b/pkl-core/src/main/java/org/pkl/core/runtime/CertificateUtils.java
index 4d7f750..eabf83a 100644
--- a/pkl-core/src/main/java/org/pkl/core/runtime/CertificateUtils.java
+++ b/pkl-core/src/main/java/org/pkl/core/runtime/CertificateUtils.java
@@ -51,11 +51,14 @@ public class CertificateUtils {
}
private static InputStream toInputStream(Object cert) throws IOException {
+ System.out.println("toInputStream! " + cert);
if (cert instanceof Path) {
var pathCert = (Path) cert;
return Files.newInputStream(pathCert);
}
if (cert instanceof InputStream) {
+ // "Here is the problem".
+ // We're returning the same input stream, but it is already closed.
return (InputStream) cert;
}
throw new IllegalArgumentException(
@@ -65,8 +68,9 @@ public class CertificateUtils {
}
private static Collection<X509Certificate> generateCertificates(InputStream inputStream)
- throws CertificateException {
+ throws CertificateException, IOException {
//noinspection unchecked
+ System.out.println("Generating certificates.. Already closed?: " + inputStream.available());
return (Collection<X509Certificate>)
CertificateFactory.getInstance("X.509").generateCertificates(inputStream);
}
I don't know where to fix this. Possible options would be:
- Ignore the certificate for the TestRunner (using
emptyList())? - Creating a new
InputStream? - Do not close the
InputStream? - Something else?! 🤷
Maybe this helps a bit 🙃