Add support for "runtime fragments"

[rhuss]

Since all generated resource objects end up in the artifact jars within META-INF/fabric8 they have to be independent of any concrete cluster installation (i.e. the must not contain the value of certain namespaces or private registries)

However, for fabric8:apply it make sense also to apply runtime specific objects which e.g. references also the namespace / project.

For example, consider role bindings, which contains the namespace in their values.

Build configs and Image Stream are other examples which are platform specific. These are not used during fabric8:apply but for fabric8:build and generated automatically. It would be nice if those could be enriched as well.

So I suggest introducing an extra runtime fragments directory, parallel to src/main/fabric8. E.g. a src/main/fabric8-rt. This could contain e.g. the role bindings required and they would be enriched when fabric8:apply or fabric8:build is called.

@jstrachan wdyt? I believe this is a good solution to keep separate and we can even add more fanciness when we are doing the deployment. E.g. we can introduce pre-defined variables (like ${namespace} or ${registry} which we then can put into the spots needed.

[jstrachan]

I'm not yet sure of the difference between the new runtime stuff and non-runtime. e.g. if a DeploymentConfig needs a RoleBinding to run, then I'd assume that should be part of the openshift.yml?

If something is environment specific (such as location of a registry or a secret or whatnot) then its probably simpler to, say, use a Template or Helm Chart to add that level of environment specific parameterisation.

One area missing though is there's no such thing as a SecretClaim or ConfigMapClaim where a purely environment specific Secret or ConfigMap is required which containers purely environment specific data - which you'd not want in the kubernetes.yml or deployment.yml. Though I guess for now we can use Template parameters for those values?

Another missing area is when using TLS with certs we should not generate a Route; as we need a controller to do that stuff for us - but we can disable Route generation already right now.

Maybe we need a way to specify template parameters in source code? e.g. if you want to store the dev / test / staging / prod values for template parameters we could maybe have some folder with environments inside so that mvn fabric8:apply / fabric8:deploy / fabric8:run could take those key/value pairs to override template parameters?

e.g. if you have a few parameters for things like, where is keycloak, where is the database URL or whatnot (assuming we're not using kubernetes service discovery here) - we may have src/main/fabric8/environments/(dev|test|staging|prod.properties) where we define those different template parameters that then overwrite the Template default values?

[rhuss]

Instantiating OpenShift Templates with different parameters during fabric8:apply would make sense. Currently, afair currently we only create the template but do not apply them (e.g. Method ApplyMojo.applyTemplates() is never called). Only for Kubernetes templates are processed locally (however I wonder whether the unprocessed Template resource object ends up in kubernetes.yml, too, so that it could be processed later when fabric8:apply kicks in whithout recreating the resources ?)

So, all in all, I'm for Template in the OpenShift case but it's not so clear to me what to do in the Kubernetes case for templates at fabric8:resource (what is what finally end up in the jar) ?

One goal imo is to keep fabric8:resource and fabric8:apply independent from each other (e.g. fabric8:apply should be able to apply already created resources only without recreating them). This important when aggregating kubernetes.yml from various dependencies. fabric8:apply of course could apply templates whit parameters (but the template should be taken from the openshift.yml, not recreated).

I'm not yet very familiar with Helm charts, but would it make sense to put them into the artefacts, too and we process these on fabric8:apply time ?

Another idea: We could introduce a special parameter syntax for the fragments (e.g. @@namespace@@ with a syntax used in the maven-invoker-plugin) and if this is present we automatically (or forced via config switch) create an OpenShift template with this as parameter (where the usual ${...} props are resolved as Maven properties during build time)

We could leave these parameters unprocessed in kubernetes.yml and then process them locally as we do now during fabric8:apply. This would also work well in aggregation scenarios when dependencies are collected in multi module setups. The drawback is that the kubernetes.yml in the jars are not directly applicable with kubectl anymore, but they propable wouldnt work anyway because of the missing context dependent information.

[jstrachan]

yeah - we used to instantiate templates in the mvn plugin; guess its not been used yet in 3.x. Or folks can apply them via kubernetesApply() in a pipeline or via oc process using the target/classes/META-INF/services/fabric8/openshift.yml

e.g. here's a few templates we're making right now and applying them with oc: https://github.com/fabric8io/fabric8-online#create-a-userteam-set-of-environments-and-services

AFAIK no OpenShift specific resources ever end up in the kubernetes.yml.

I guess the kubernetes equivalent to a Template would be a Helm Chart which has similar parameterisation - though kinda better as it keeps track of previous parameter values when doing updates. e.g. you can upgrade a Chart to a new version without worrying about re-specifying previous parameter values.

Another alternative could be - if its just about parameterising environment variables - to use a ConfigMap and have a way to override values in the ConfigMap on a per environment basis (or just assume a user will do that separately).

On the helm front; am not sure we need to put the helm chart inside the jars since helm charts are a versioned distribution already (they are versioned tarballs of parameterized yamls with parameters and metadata etc). We could add a mvn fabric8:helm-apply command I guess; though the helm CLI is pretty easy to use too.

I guess if folks want parameterisation and wanna work with kubernetes and openshift then we should probably promote Helm Charts for the kubernetes case; they work really well for that. Though I guess we could maybe also generate a kubernetes-template.yml too which isn't usable via kubectl but could be processed via mvn fabric8:apply or via oc process and then applied via kubectl?

[kameshsampath]

@ro14nd not sure this makes more contextual here, but it would be great to have BuildConfigs as well part of the generation/apply, as at times its become essential that we use jenkins pipeline for our application but that has to be done manually. may be something like bc.yaml in our project creates and uses that build configuration

[jstrachan]

@kameshsampath I saw this come up on IRC recently and never had chance to dive into the use case. So the fabric8:resource goal builds resources, at build time, to run an app. Creating a BuildConfig is a bit late as usually the build is already running when the fabric8:resource goal is invoked ;) Why would a build (running in a BuildConfig) want to generate a new BuildConfig? I don't quite grok it. Unless you mean the S2I BuildConfig thats generated as part of a fabric8:build when running a Jenkins pipeline? If so that already works.

Or is it just case of wanting to import a git repo into OpenShift as a pipeline? mvn fabric8:import might be something we could make more flexible - it used to be quite gogs centric but maybe we can swizzle it to be more of a BuildConfig generator? https://maven.fabric8.io/#fabric8:import

[kameshsampath]

@jstrachan may am wrong understanding a bit here .. but what i was trying to know - how to create the build config objects via f8-m-p, though the phases are different my thought was around how-to, if f8:import does then let me give shot at it and see what it does and see what we might need

[jstrachan]

so try use mvn fabric8:import and it should do the trick. We may need to tweak the generated BuildConfig a little to properly use the new jenkinsPipelineStrategy if it detects a Jenkinsfile though.

e.g. if its already in git then it just tries to create a BC to refer to the git repo: https://github.com/fabric8io/fabric8-maven-plugin/blob/master/plugin/src/main/java/io/fabric8/maven/plugin/mojo/internal/ImportMojo.java#L162

if its not in git then it first tries to do the project creation in, say, gogs.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[rhuss]

@fabric8io/fmp-core-team should we pick up the discussion again ? After we have gained now some experiences over the years, I think the it's still valid, that there are two kinds of deployment configuration: One which is independent on the target platform where the application is deployed (like deployments, services, etc.) and some which are very specific of the final cluster where the application is deployed to (best example is the "namespace"). So, should the "namespace" part of the generated resource descriptors which are stored also in the jars and distributed via Maven or should it be specified somewhere else (these kind of runtime fragments, or in the configuration for 'fabric8:apply'). I think it should be separated.

For other things like configmaps or secrets which differ from environment to environment its similar but not so clear. I really think that configmaps should be added only as templates to the generated resources, which are filled when applied.

It's a larger, principal discussion in which direction f-m-p should go, so I don't expect a quick solution.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!