[component pipeline] Multi cluster deploy

[michaelsauter]

Closes https://github.com/opendevstack/ods-jenkins-shared-library/issues/496

Work in progress.

The description of the feature is located at the bottom of https://github.com/opendevstack/ods-jenkins-shared-library/blob/bfcd08a2d21e7fe07d551fcb154c1b815481cc68/docs/modules/jenkins-shared-library/partials/odsComponentStageRolloutOpenShiftDeployment.adoc.

I would appreciate your feedback on general idea, configuration format, open questions you have, etc. @fkressmann @SHoen @renedupont @gerardcl @henrjk @clemensutschig @metmajer

[michaelsauter]

Update: I realised today that the configuration for external clusters can be simplified a lot, because we already have the source and target credentials by looking at the serviceaccounts :) This means pulling does not really make sense (it is not safer than pushing) and we only need to specify apiUrl, apiCredentials and registryHost. No need to create and link secrets. I've updated the code base and the documentation accordingly.

[fkressmann]

Nice! Seems it will not be necessary anymore to habe two OpenShift configurations for build and deploy cluster anymore :) ~~Still, one question: Usually, all components in one project would have the same ods-environments configuration. In case something changes, every Jenkinsfile would need to be changed. Maybe there is an option to have this configuration project-global.~~ Nevermind, understood the docs wrong 🙄

[michaelsauter]

@fkressmann Thanks for the feedback.

I realised that the name of the target would be really helpful to use e.g. when specifying Tailor (or future Helm) parameter / values files. Otherwise it would not be possible to use different parameters per target. To this end, I added 297bde4. @fkressmann would this help your use case? Do you take more decision on the target name which could not be implemented using the env var?

[fkressmann]

I think it's a step in the right direction. What we do is selecting the .env files for tailor using (our equivalent of) the target name and passing it as tailorParamFile because the config in the .env files differs between the target names (for example database connections...) How would you do this at the moment?

[michaelsauter]

@fkressmann With my change you could write this:

odsComponentStageRolloutOpenShiftDeployment(context, [tailorParamFile: '${TARGET}.env'])

Note the single quotes - that means Groovy does not evaluate this as a variable, but it will get evaluated by the shell later on, which executes sth. like:

tailor -n foo --param-file ${TARGET}.env apply

Around the tailor invocation, the shell variable TARGET is set to the target name (such as local in the example).

[michaelsauter]

Next to my proposal as linked above, I think there is another way to achieve the same result. I want to share it here to see what you all think ...

The proposal above has the nice property that one does not need to modify the Jenkinsfile a lot. However, in reality, at least one commit to each repo will be necessary as each target likely needs its own parameters. If we need a commit anyway, we could also require a change to the Jenkinsfile. For the given example, it could look like this:

@Library('ods-jenkins-shared-library@latest') _
odsComponentPipeline(
  imageStreamTag: 'ods/jenkins-agent-base:latest'
) { context ->
  odsComponentFindOpenShiftImageOrElse(context) {
    odsComponentStageBuildOpenShiftImage(context)
  }

  odsComponentStageRolloutOpenShiftDeployment(context, [branch: 'master', namespace: 'foo-dev'])

  odsComponentStageRolloutOpenShiftDeployment(context, [branch: 'release/', namespace: 'foo-test'])
  odsComponentStageRolloutOpenShiftDeployment(context, [branch: 'release/', namespace: 'foo-uat'])
  odsComponentStageRolloutOpenShiftDeployment(context, [
    branch: 'release/',
    namespace: 'foo-test',
    apiUrl: 'https://api.a3pqde7f.westeurope.aroapp.io:6443',
    apiCredentialsSecret: 'aro-api',
    registryHost: 'image-registry.apps.a3pqde7f.westeurope.aroapp.io'])
}

The advantage of this would be:

• greater freedom how to configure the options for each target
• less complexity in the rollout stage
• no ConfigMap needed ... that also makes it easier to see where deployment happens by just looking at this file, plus the target configuration is version-controlled

The disadvantage I see:

• no branchToEnvironmentMapping anymore needed, but this makes it hard to know for which branches to build a container image. Also, it looses any notion of an "environment" - even though we do not use this anywhere I know of right now, this could be a helpful concept to have.
• API URL and registry host would be repeated across Jenkinsfiles ... potentially this could be overcome by allowing to reference a ConfigMap that specifies target cluster attributes

There could also be a version of this which still makes use of branchToEnvironmentMapping:

@Library('ods-jenkins-shared-library@latest') _
odsComponentPipeline(
  imageStreamTag: 'ods/jenkins-agent-base:latest',
  branchToEnvironmentMapping: [
    'master': 'dev',
    'release/': 'test'
  ]
) { context ->
  odsComponentFindOpenShiftImageOrElse(context) {
    odsComponentStageBuildOpenShiftImage(context)
  }

  odsComponentStageRolloutOpenShiftDeployment(context, [environment: 'dev'])

  odsComponentStageRolloutOpenShiftDeployment(context, [environment: 'test'])
  odsComponentStageRolloutOpenShiftDeployment(context, [environment: 'test', namespace: 'foo-uat'])
  odsComponentStageRolloutOpenShiftDeployment(context, [
    environment: 'test',
    apiUrl: 'https://api.a3pqde7f.westeurope.aroapp.io:6443',
    apiCredentialsSecret: 'aro-api',
    registryHost: 'image-registry.apps.a3pqde7f.westeurope.aroapp.io'])
}

Any thoughts on this? Other ideas?

[gerardcl]

I like a lot this last approach, since:

• by default the jenkinsfile one gets provisioned it is only using master -> dev indeed
• an environment (in ODS) is at the end an artificial concept that just defines the context where a specific version/stage of your application is deployed/resides (which can be a different namespace, a different cluster and/or different previews in the same namespace...)
• it keeps the chance of being able to fast implement feature previews based on using OpenShift templates as stated in the ODS docs already
• as you already mentioned, it is really transparent and exposed to the Jenkinsfile

[metmajer]

@michaelsauter and I identified some important questions, documented here: https://github.com/opendevstack/ods-jenkins-shared-library/issues/496#issuecomment-819428222