Fixes #214, adds support for using job-specific policies

[bluesliverx]

This adds support for separating job policies as described in #214. This is inspired by the Salt integration and their usage of per-minion policies. The vault plugin may be configured with optional "policies" that may be templatized with tokens that are unique to the job. This allows more isolation of job permissions in Vault, thereby allowing for more secure Jenkins configurations.

I was able to add most of the logic for this functionality into the expiring credential base class. I also had to separate the cached token value by the list of policies specified so that different lists of policies resulted in new tokens. I have tested this on our Jenkins environment and it has allowed me to completely separate the policies of two jobs so that they can only read specific paths in Vault. I also added unit testing for all functionality I added (AFAIK, let me know if there is any missing).

I opted for two configuration options instead of a single one, since in our environment we need to migrate from one method to the other (i.e. AppRole based auth that has all permissions vs AppRole based auth that enables using the limited tokens). This is done in a separate commit in case there are disagreements if this should be how it is configured or not. Without this option, as soon as I enabled policies in the folder or global configuration, the existing auth method no longer worked properly. This may not be useful outside of our use case, so please let me know if I need to remove it.

• [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• [x] Ensure that the pull request title represents the desired changelog entry
• [x] Please describe what you did
• [x] Link to relevant issues in GitHub or Jira
• [x] Link to relevant pull requests, esp. upstream and downstream changes
• [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue

[bluesliverx]

RETEST

[jetersen]

@bluesliverx not sure what this achieves? Wouldn't it be relatively easy to copy a job and remove the use policy and it continues to work.

This is only implemented on the plugin side, nothing on hashicorp vault enforces this.

Or am I mistaken?

[bluesliverx]

@jetersen sorry, I've been vacationing for a couple of weeks.

Yes, there are some security flaws in this depending on the job configuration. I was thinking I should probably remove the ability to configure this on the job itself so that people can't override it at that level. Thoughts?

The idea here is that there is a structure of folders/jobs which can be utilized to segment the vault policies. This limits the ability of one job to read the secrets that should only be read by another job. This is the same methodology used in Salt and works very well there. I liked how it was done and wanted to extend the Vault plugin to do the same thing.

For a little background, my team manages a Jenkins cluster that has thousands of jobs across different folders. Currently we just allow all jobs to read secrets based on a single Jenkins policy, but we really want to segment that so that we limit security holes in the system. This would allow us to do that.

[bluesliverx]

@jetersen, any feedback on this?

[bluesliverx]

I fixed a bug that we identified when we changed the TTL on the approle auth. The child tokens were still created with their default TTL, which was much longer than the approle-based token, and once it expired the child tokens were expired but they were still attempted to be used. I now set the child token TTL to the parent token TTL.

[bluesliverx]

@jetersen thanks for taking a look at this again. In addition to the code review changes, I also implemented a disable flag for sub-folders and jobs, which should prevent escalation of policies on rogue folders/jobs when the admin enables it. Let me know what you think!

[bluesliverx]

I'm happy to do a separate PR for that if you want. We run a decently large build grid (15k+ jobs with likely hundreds of runs each hour) and I haven't noticed any slowdowns to any of those jobs based on retrieving Vault secrets, just FYI.