BuildConfig cicd-pipeline doesn't work with jenkins-persistent template

[boeboe]

Hi all,

I have tried to switch from ephemeral jenkins to persistent jenkins. For this, I modified the jenkins-persistent template in de openshift project to point to a PersistentVolumeClaim backed by heketi. I also added the following configuration at the bottom of my master-config as instructed in the documentation to override the default template to be used for a BuildConfig type JenkinsPipeline.

jenkinsPipelineConfig:
  autoProvisionEnabled: true 
  templateNamespace: openshift 
  templateName: jenkins-persistent 
  serviceName: jenkins

My modified template looks like the following (adding storageClassName: heketi):

apiVersion: template.openshift.io/v1
kind: Template
labels:
  template: jenkins-persistent-template
message: >-
  A Jenkins service has been created in your project.  Log into Jenkins with
  your OpenShift account.  The tutorial at
  https://github.com/openshift/origin/blob/master/examples/jenkins/README.md
  contains more information about using this template.
metadata:
  annotations:
    description: >-
      Jenkins service, with persistent storage.


      NOTE: You must have persistent volumes available in your cluster to use
      this template.
    iconClass: icon-jenkins
    openshift.io/display-name: Jenkins (Persistent)
    tags: 'instant-app,jenkins'
    template.openshift.io/documentation-url: 'https://docs.openshift.org/latest/using_images/other_images/jenkins.html'
    template.openshift.io/long-description: >-
      This template deploys a Jenkins server capable of managing OpenShift
      Pipeline builds and supporting OpenShift-based oauth login.
    template.openshift.io/provider-display-name: 'Red Hat, Inc.'
    template.openshift.io/support-url: 'https://access.redhat.com'
  creationTimestamp: '2017-10-19T10:55:00Z'
  name: jenkins-persistent
  namespace: openshift
  resourceVersion: '103975'
  selfLink: >-
    /apis/template.openshift.io/v1/namespaces/openshift/templates/jenkins-persistent
  uid: f44bb359-b4bb-11e7-b702-00505683db64
objects:
  - apiVersion: v1
    kind: Route
    metadata:
      annotations:
        template.openshift.io/expose-uri: 'http://{.spec.host}{.spec.path}'
      name: '${JENKINS_SERVICE_NAME}'
    spec:
      tls:
        insecureEdgeTerminationPolicy: Redirect
        termination: edge
      to:
        kind: Service
        name: '${JENKINS_SERVICE_NAME}'
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: '${JENKINS_SERVICE_NAME}'
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: '${VOLUME_CAPACITY}'
      storageClassName: heketi
  - apiVersion: v1
    kind: DeploymentConfig
    metadata:
      annotations:
        template.alpha.openshift.io/wait-for-ready: 'true'
      name: '${JENKINS_SERVICE_NAME}'
    spec:
      replicas: 1
      selector:
        name: '${JENKINS_SERVICE_NAME}'
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            name: '${JENKINS_SERVICE_NAME}'
        spec:
          containers:
            - capabilities: {}
              env:
                - name: OPENSHIFT_ENABLE_OAUTH
                  value: '${ENABLE_OAUTH}'
                - name: OPENSHIFT_ENABLE_REDIRECT_PROMPT
                  value: 'true'
                - name: OPENSHIFT_JENKINS_JVM_ARCH
                  value: '${JVM_ARCH}'
                - name: KUBERNETES_MASTER
                  value: 'https://kubernetes.default:443'
                - name: KUBERNETES_TRUST_CERTIFICATES
                  value: 'true'
                - name: JNLP_SERVICE_NAME
                  value: '${JNLP_SERVICE_NAME}'
              image: ' '
              imagePullPolicy: IfNotPresent
              livenessProbe:
                failureThreshold: 30
                httpGet:
                  path: /login
                  port: 8080
                initialDelaySeconds: 420
                timeoutSeconds: 3
              name: jenkins
              readinessProbe:
                httpGet:
                  path: /login
                  port: 8080
                initialDelaySeconds: 3
                timeoutSeconds: 3
              resources:
                limits:
                  memory: '${MEMORY_LIMIT}'
              securityContext:
                capabilities: {}
                privileged: false
              terminationMessagePath: /dev/termination-log
              volumeMounts:
                - mountPath: /var/lib/jenkins
                  name: '${JENKINS_SERVICE_NAME}-data'
          dnsPolicy: ClusterFirst
          restartPolicy: Always
          serviceAccountName: '${JENKINS_SERVICE_NAME}'
          volumes:
            - name: '${JENKINS_SERVICE_NAME}-data'
              persistentVolumeClaim:
                claimName: '${JENKINS_SERVICE_NAME}'
      triggers:
        - imageChangeParams:
            automatic: true
            containerNames:
              - jenkins
            from:
              kind: ImageStreamTag
              name: '${JENKINS_IMAGE_STREAM_TAG}'
              namespace: '${NAMESPACE}'
            lastTriggeredImage: ''
          type: ImageChange
        - type: ConfigChange
  - apiVersion: v1
    kind: ServiceAccount
    metadata:
      annotations:
        serviceaccounts.openshift.io/oauth-redirectreference.jenkins: >-
          {"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"${JENKINS_SERVICE_NAME}"}}
      name: '${JENKINS_SERVICE_NAME}'
  - apiVersion: v1
    groupNames: null
    kind: RoleBinding
    metadata:
      name: '${JENKINS_SERVICE_NAME}_edit'
    roleRef:
      name: edit
    subjects:
      - kind: ServiceAccount
        name: '${JENKINS_SERVICE_NAME}'
  - apiVersion: v1
    kind: Service
    metadata:
      name: '${JNLP_SERVICE_NAME}'
    spec:
      ports:
        - name: agent
          nodePort: 0
          port: 50000
          protocol: TCP
          targetPort: 50000
      selector:
        name: '${JENKINS_SERVICE_NAME}'
      sessionAffinity: None
      type: ClusterIP
  - apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.alpha.openshift.io/dependencies: '[{"name": "${JNLP_SERVICE_NAME}", "namespace": "", "kind": "Service"}]'
        service.openshift.io/infrastructure: 'true'
      name: '${JENKINS_SERVICE_NAME}'
    spec:
      ports:
        - name: web
          nodePort: 0
          port: 80
          protocol: TCP
          targetPort: 8080
      selector:
        name: '${JENKINS_SERVICE_NAME}'
      sessionAffinity: None
      type: ClusterIP
parameters:
  - description: The name of the OpenShift Service exposed for the Jenkins container.
    displayName: Jenkins Service Name
    name: JENKINS_SERVICE_NAME
    value: jenkins
  - description: The name of the service used for master/slave communication.
    displayName: Jenkins JNLP Service Name
    name: JNLP_SERVICE_NAME
    value: jenkins-jnlp
  - description: >-
      Whether to enable OAuth OpenShift integration. If false, the static
      account 'admin' will be initialized with the password 'password'.
    displayName: Enable OAuth in Jenkins
    name: ENABLE_OAUTH
    value: 'true'
  - description: Whether Jenkins runs with a 32 bit (i386) or 64 bit (x86_64) JVM.
    displayName: Jenkins JVM Architecture
    name: JVM_ARCH
    value: i386
  - description: Maximum amount of memory the container can use.
    displayName: Memory Limit
    name: MEMORY_LIMIT
    value: 512Mi
  - description: 'Volume space available for data, e.g. 512Mi, 2Gi.'
    displayName: Volume Capacity
    name: VOLUME_CAPACITY
    required: true
    value: 1Gi
  - description: The OpenShift Namespace where the Jenkins ImageStream resides.
    displayName: Jenkins ImageStream Namespace
    name: NAMESPACE
    value: openshift
  - description: Name of the ImageStreamTag to be used for the Jenkins image.
    displayName: Jenkins ImageStreamTag
    name: JENKINS_IMAGE_STREAM_TAG
    value: 'jenkins:latest'

However... the jenkins POD's never show up and somehow seem never to get trigger to do so as well. I am quite sure heketi is working because I also modified all persistentVolumeClaim's in this demo to heketi and that works well.

Any idea on how to debug this or where to start this demo with persistent storage for jenkins?

Best regards, Bart

[boeboe]

Please close the issue... the reason was that there were still some jenkins related services not cleaned-up (remaining artefacts from the ephemeral experiment within the same project) and I found out in the documentation that the BuildConfig type JenkinsPipeline needs uniqueness on the service to (re)-initiate.