Support decrypting credentials using an external certificate (aka "make secrets portable")

[oleg-nenashev]

As a user I want to share a single configuration file between multiple Jenkins instance, including credential definitions. Currently JCasC support plugin supports defining encrypted secrets on the configuration YAML. Configuration example:

credentials:
  system:
    domainCredentials:
    - credentials:
      - usernamePassword:
          id: "exampleuser-creds-id"
          username: "exampleuser"
          password: "{AQAAABAAAAAQ1/JHKggxIlBcuVqegoa2AdyVaNvjWIFk430/vI4jEBM=}"
          scope: GLOBAL

Encryption is done using the Jenkins-internal secret key which is unique for every Jenkins instance. It means that the credentials are not portable between instances. It also creates obstacles for immutable images which start with a fresh Jenkins instance and initially do not have an initialized secret key for encryption. Although there are workarounds, I suggest adding support of external certificates.

Proposal:

• Users can refer external credentials using a custom string, e.g. {ENC, PKCS7,AQAAABAAAAAQ1/JHKggxIlBcuVqegoa2AdyVaNvjWIFk430/vI4jEBM=} (encryptted text)
• Encryption keys can be passed through a file. Path to it can be defined via environment variable or the JCasC context configuration section
• Nice2Have: Arbitrary encryption engines are supported, maybe using an extension point

Implementation notes:

• The logic can be implemented using a new SecretSource class which includes underlying extensions for encryption methods

[oleg-nenashev]

@timja @jonbrohauge This is what we were talking about on Oct at the end of the meeting

[oleg-nenashev]

OKay, I have finally forgot about it. Sorry all. Once I get back to JCasC, getting the patch over the line will be my top priority

[qalinn]

Any update on this issue?

[oleg-nenashev]

Nope. I was unable to finish it due to COVID-19 and other emergencies. No ETA, sorry

On Wed, Jul 1, 2020, 11:22 qalinn [email protected] wrote:

Any updated on this issue?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/jenkinsci/configuration-as-code-plugin/issues/1141#issuecomment-652302685, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW4RIFZK2DYJR72URLS3NDRZL56LANCNFSM4I7MQIMQ .

[oleg-nenashev]

Better late than never, working on it again. Thanks to recent patches by @jetersen , now we have a common way to extend variable resolution methods.

[dhs-rec]

I like the idea. Looks similar to what eyaml provides for Puppet's Hiera (looks like ENC[PKCS7, encrypted text] there). But eyaml in itself is just a framework which can be enhanced with different encryption backends (like Vault, GnuPG, KMS,...). We're using it with the GnuPG backend (ENC[GPG, encrypted text]) since it offers the most flexibility. Encrypting the secrets with multiple keys at once allows them to be decrypted/edited by multiple users, using their own keys, and also to reuse them on several installations, which can also have their own keypairs each. eyaml also comes with a handy command line tool which allows for easy re-encryption in case of key changes.

Would be nice to have similar functionality available here, too.

[rgarrigue]

Hi

Until it's done, can you give pointers to the alternative methods ? I'm looking for a way to migrate secrets from an old to a new Jenkins. A one time operation. Can I force Jenkins' internal secret key for example ?

Thanks,

[jazzbeaux59]

Hi

Until it's done, can you give pointers to the alternative methods ? I'm looking for a way to migrate secrets from an old to a new Jenkins. A one time operation. Can I force Jenkins' internal secret key for example ?

Thanks,

Yes, alternatives would be appreciated.

[rgarrigue]

We came up with a couple of groovy script to export credentials in the JCasC format. Here they are, for anyone interested.

Note, they fit our use case with our Username & Password + String + File + SSH credentials, if you've additional kind of credentials you'll need to add stuff.

def creds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials()
def credsFile = new File('/tmp/secrets/all-secrets.yaml')
for(c in creds) {
  if(c instanceof com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey){
    yaml = String.format(
'''\
            - basicSSHUserPrivateKey:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                username: "%s"
                privateKeySource:
                  directEntry:
                    privateKey: "%s"
''',
      c.id,
      c.description,
      c.username,
      c.privateKeySource.getPrivateKeys()[0],
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl){
    yaml = String.format(
'''\
            - usernamePassword:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                username: "%s"
                password: "%s"
''',
      c.id,
      c.description,
      c.username,
      c.password,
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl){
    yaml = String.format(
'''\
            - string:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                secret: "%s"
''',
      c.id,
      c.description,
      c.secret,
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof  org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
    yaml = String.format(
'''\
            - file:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                fileName: "%s"
                secretBytes: "%s"
''',
      c.id,
      c.description,
      c.fileName,
      c.secretBytes.plainData.encodeBase64(),
    )
    print(yaml)
    credsFile.append(yaml)
  }
}

And we needed a 2nd one for a sub cred domain "Debian package builder"

def domainCreds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getDomainCredentials()
for (domainCred in domainCreds) {
  if (domainCred.domain.name != "Debian package builder") {
    continue
  }
  def credsFile = new File('/tmp/secrets/builder.yaml')
  for (c in domainCred.getCredentials()) {
  if (c instanceof  org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
    yaml = String.format(
'''\
            - file:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                fileName: "%s"
                secretBytes: "%s"
''',
      c.id,
      c.description,
      c.fileName,
      c.secretBytes.plainData.encodeBase64(),
    )
    print(yaml)
    credsFile.append(yaml)
  }
}

[philippfe]

Hi, is there an Update for an ETA? Cheers, Philipp

[spqr2001]

Hi , would be nice to have that feature.. cheers Michael

[dhs-rec]

Since JCasC can use environment variables to fill in credentials, we've solved this by switching to Vault as an external credentials provider (needs Vault plugin). If setup properly, JCasC can then read its initial (Vault approle) credentials from Vault itself (yes, sounds weird), treated as environment variables. The setup looks like this (assuming your Jenkins version already comes with systemd service file):

1. Setup Vault incl. approle login provider and kv2 secrets store
2. Create a jenkins approle and setup appropriate access policies
3. Create a secret secret/jenkins/jcasc in the k/v store containing two k/v pairs: [VAULT_ROLE_ID, <your_role_id>], [VAULT_SECRET_ID, <your_secret_id>]
4. Create a file /var/lib/jenkins/vault with the following content (make sure to protect it properly):

CASC_VAULT_APPROLE=<your role id>
CASC_VAULT_APPROLE_SECRET=<your secret id>
CASC_VAULT_PATHS=secret/jenkins/jcasc
CASC_VAULT_URL=<vault url>

5. Create a service overlay file in /etc/systemd/system/jenkins.service.d/ (name doesn't matter, but must have a .conf extension), with the following content:

[Service]
Environment="CASC_VAULT_FILE=/var/lib/jenkins/vault"

6. Run systemctl daemon-reload and systemctl restart jenkins.service
7. Add the following to your JCasC credentials: section (the variables should match the k/v pairs from step 3 above):

credentials:
  system:
    domainCredentials:
    - credentials:
      - vaultAppRoleCredential:
          description: "Jenkins credentials for accessing Vault"
          id: "JenkinsApprole"
          path: "approle"
          roleId: "${VAULT_ROLE_ID}"
          scope: SYSTEM
          secretId: "${VAULT_SECRET_ID}"

8. You can then migrate your credentials into Vault and reference them from JCasC using something like:

      - vaultUsernamePasswordCredentialImpl:
          description: "Some User"
          engineVersion: 2
          id: "SOME_USER"
          passwordKey: "password"
          path: "secret/jenkins/some_user"
          scope: GLOBAL
          usernameKey: "username"
      - vaultSSHUserPrivateKeyImpl:
          description: "Some User (SSH)"
          engineVersion: 2
          id: "SOME_USER_SSH"
          passphraseKey: "key_passphrase"
          path: "secret/jenkins/some_user"
          privateKeyKey: "private_key"
          scope: GLOBAL

NOTE: The Vault plugin doesn't support all credential types, yet (AWS for example) and there are also some plugins which don't use the Jenkins credential system at all. In this case you can still work around this by adding more variables to secret/jenkins/jcasc and reference those just as we did in step 7.

[ukuko]

We came up with a couple of groovy script to export credentials in the JCasC format. Here they are, for anyone interested.

Note, they fit our use case with our Username & Password + String + File + SSH credentials, if you've additional kind of credentials you'll need to add stuff.

def creds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials()
def credsFile = new File('/tmp/secrets/all-secrets.yaml')
for(c in creds) {
  if(c instanceof com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey){
    yaml = String.format(
'''\
            - basicSSHUserPrivateKey:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                username: "%s"
                privateKeySource:
                  directEntry:
                    privateKey: "%s"
''',
      c.id,
      c.description,
      c.username,
      c.privateKeySource.getPrivateKeys()[0],
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl){
    yaml = String.format(
'''\
            - usernamePassword:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                username: "%s"
                password: "%s"
''',
      c.id,
      c.description,
      c.username,
      c.password,
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl){
    yaml = String.format(
'''\
            - string:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                secret: "%s"
''',
      c.id,
      c.description,
      c.secret,
    )
    print(yaml)
    credsFile.append(yaml)
  }
  if (c instanceof  org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
    yaml = String.format(
'''\
            - file:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                fileName: "%s"
                secretBytes: "%s"
''',
      c.id,
      c.description,
      c.fileName,
      c.secretBytes.plainData.encodeBase64(),
    )
    print(yaml)
    credsFile.append(yaml)
  }
}

And we needed a 2nd one for a sub cred domain "Debian package builder"

def domainCreds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getDomainCredentials()
for (domainCred in domainCreds) {
  if (domainCred.domain.name != "Debian package builder") {
    continue
  }
  def credsFile = new File('/tmp/secrets/builder.yaml')
  for (c in domainCred.getCredentials()) {
  if (c instanceof  org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
    yaml = String.format(
'''\
            - file:
                scope: "GLOBAL"
                id: "%s"
                description: "%s"
                fileName: "%s"
                secretBytes: "%s"
''',
      c.id,
      c.description,
      c.fileName,
      c.secretBytes.plainData.encodeBase64(),
    )
    print(yaml)
    credsFile.append(yaml)
  }
}

Hi, would you please add details on how to use this on the end side (import)? I guess you use docker swarm?

[ukuko]

hi, is there any ETA for this?

[gildor7]

Hi, I have the same issue, some news?

[timja]

No one is actively working on this.