aws-secrets-manager-credentials-provider-plugin
aws-secrets-manager-credentials-provider-plugin copied to clipboard
jenkinsci
AWS Secrets Manager credentials provider plugin can't retrieve secret from a http request in jenkins job
Hello Team, In my organization we have started using the AWS Secrets Manager plugin to replace storing our secrets in Jenkins. We have a job which performs a http request, the challenge I'm facing with this Plugin is the credentials can't be retrieved while making a httpRequest call containing the secret in the request and throws a serialization error. It works fine when we use credentials('secret-value-to-retrieve') but not from authentication('secret-value-to-retrieve') based on the below groovy dsl.
Appreciate any suggestions.
steps {
httpRequest { url('apiEndpoint') httpMode('POST') authentication('secret-value-to-retrieve') acceptType('APPLICATION_JSON') contentType('APPLICATION_JSON') requestBody('sample Body') validResponseCodes('201,307') consoleLogResponseBody(false) } }
Exception: FATAL: Unable to serialize jenkins.plugins.http_request.HttpRequestExecution@6d5ace62
java.io.NotSerializableException: io.jenkins.plugins.credentials.secretsmanager.factory.CredentialsFactory$SecretSupplier
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at hudson.remoting.UserRequest._serialize(UserRequest.java:263)
at hudson.remoting.UserRequest.serialize(UserRequest.java:272)
Caused: java.io.IOException: Unable to serialize jenkins.plugins.http_request.HttpRequestExecution@6d5ace62
at hudson.remoting.UserRequest.serialize(UserRequest.java:274)
at hudson.remoting.UserRequest.
Are you running Jenkins in distributed mode (is this build running on a different box from the master)? We have seen some serialisation errors before like this.
@chriskilding - Yes correct , the build is running on a slave node not on master. Due to think authentication('secret-value-to-retrieve') can retrieve the credentials using the plugin ?. Is that an issue ? do you have any suggestions.
I've never used the http request plugin before, so just trying to work a few things out about it.
I can see the authentication argument is used to query the Credentials API in the plugin, and it acts differently depending on what credential type is used. (Username with password = basic authentication, Certificate = certificate authentication.) If this lookup happens on a build agent rather than the master, then I can see how serialisation could fail; hopefully this should be a simple enough fix on the SecretSupplier class.
I don't see a credentials argument within the plugin. Do you just mean that it works when you use the general credentials binding system?
Yes correct it does work when it's used in the credentials binding and also in the below example. The SecretSupplier fix you mentioned is it in the plugin ?. What would the ETA for the fix to be released ? .
job('example-2') { scm { git { remote { github('exampleGithubRep) credentials('gihub_key') } } } }
Ok I've looked in more detail and it's not quite as simple as I thought at first glance...
The reason that withCredentials or credentials invocations in Jenkins Pipeline work, is that they take a snapshot of the credential on the master node at the start of a job. They hand the credential snapshot to the build agent, and the snapshot is what's used throughout the job.
The stack trace you're seeing is the HTTP request plugin trying to use the credentials API from a build agent box, without taking a snapshot of the credential. Because no snapshot was taken, the live AWS-backed version of the credential was accessed and serialised. This failed because the AWS-backed credential contains an instance of the AWS SDK, which is never serialisable.
So I think the HTTP request plugin is the thing that needs a patch, to take a snapshot of a credential before using it when it's operating in remote mode.
I've reported this on the HTTP Request Plugin's issue tracker here: https://issues.jenkins.io/browse/JENKINS-66118