git + branch as configuration source

[ewelinawilkosz]

trafficstars

we now support folder as a configuration source, what I'm missing is git repository + branch as a location. it would allow e.g. multiple teams to share common configuration and maintain instance specific on a branch. so plugin would take two args - repo & branch and read yaml from repo root folder

any opinions?

[jonbrohauge]

So you say that JCasC should download one or more Jenkins configuration yaml-files from the repository, or should it read the yaml as a stream, i.e. opening up for the possibility of having a Consul-like service that gives you a yaml-based response?

[ewelinawilkosz]

I'm just saying I want to have a possibility to have multiple jenkins instances configured with files stored in one repo, where each branch represents different jenkins. that way we can maintain common configuration on e.g. master, and instance specific in files added on branches. if there is a change on master you merge to your branch and get changes in let's say jenkins_common.yaml but your instance specific yamls are not touched.

how we do it is to be decided - do you have preferred solution?

[jonbrohauge]

Cool, I like that. Just wondering.

A possible long term goal could be when a change has been detected on the branch that your Jenkins instance is watching, it'll do a reload. Thinking of a push-trigger from your git repository.

[ewelinawilkosz]

definitely, but I would also like to allow users to configure jcasc behavior - do they want autoreload or do they want to control it (we have an issue for that #274) anyway, as you say, long term probably - we'll try to include it in 1.0 but no promises

[Mathiasdm]

It would be nice to have this in a version-control independent manner, to support for example Mercurial.

[x10an14]

@ewelinawilkosz I very much like this idea =)

I'm just saying I want to have a possibility to have multiple jenkins instances configured with files stored in one repo, where each branch represents different jenkins. that way we can maintain common configuration on e.g. master, and instance specific in files added on branches. if there is a change on master you merge to your branch and get changes in let's say jenkins_common.yaml but your instance specific yamls are not touched.

Makes it easier for a platform/sysadmin -team to maintain a "dev" branch of Jenkins yaml file(s), whilst the "production" Jenkins instances refer to the master branch by default =D

Would it be possible to make this git-host agnostic too? My impression is that currently in 0.10-alpha, most, if not all Git-config is relying on you to use Github specifically. (We're running Bitbucket unfortunately).

[ewelinawilkosz]

I get more and more requests for that one so I'm changing the priority

[HerrmannHinz]

i approached something similar with a little hack:

• https://github.com/jimmidyson/configmap-reload

SCENARIO:

• yaml files get persisted/provided via a kubernetes configmap as a file mount
• we did spawn the above container (jimmidyson/configmap-reload) as a sidecar to the jenkins instance. it has the same configmap mounted and watches it.
• whenever the configmap has been changed (configmap-reloader is watching this), the reloader will trigger the https://jenkins/configuration-as-code/reload endpoint.

maybe this is helpful for some ppl.

[torstenwalter]

@HerrmannHinz How did you handle authentication for the reload endpoint? Did you use a pre-provisioned token?

[jetersen]

Perhaps we could use git-client-plugin. Since both GitHub, Gitlab, and bitbucket supports

git clone https://{username}:{}@github.com/owner/repo.git
git clone https://{username}:{private_token}@gitlab.com/owner/repo.git
git clone https://{username}:{access_token}@bitbucket.org/user/repo.git
git clone https://x-token-auth:{access_token}@bitbucket.org/user/repo.git

This works for bitbucket server as well

Of course, requires git on Jenkins master. It could replace accessing files on via REST/RAW web requests. Which is partly done in #665

I know for a fact that accessing Gitlab git files/folders has to be done via REST API, they no longer support reading from git raw links with API token. https://gitlab.com/gitlab-org/gitlab-ce/issues/55081.

With git clone, you can even check out a single branch, but you can also check out to a specific directory as well.

[pallam-bigfoot]

Can we use GitHub link as configuration source. I am getting the error as shown in the attached screenshot. I have established connection between this jenkins instance and our GitHub using a Personal Access Token. Can you please let me know if i am doing something wrong here.

[jetersen]

we do not support authentication, you could consider making your config public if you have handled secrets correctly.

[waveywaves]

I would like to start working on this. Any way this can be assigned to me. Would like to do this as part of hacktoberfest.

[Mo0rBy]

Is there any news on this being implemented? There is absolutely no way I can have my repositories publicly accessible, so for now I am resorting to using kubectl cp to copy my casc.yaml into my deployed Jenkins container.| This is assuming this issue is about implementing authentication for git repository links, not sure if that what this issue was originally intended for but I can see it being mentioned throughout this thread.

[timja]

if you are using Kubernetes why not use the helm chart to manage it? https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins#configuration-as-code

[Mo0rBy]

I am currently looking at setting up the Helm chart and I did see the relevant section in there, but with how our JobDSL stuff is configured, I think it makes more sense to have the config file within one of those repos, then it has the config + all our job definitions. It is certainly an option though.