Add support for Vault Agent

[jeffbyrnes]

SUMMARY

• Add ability to run vault agent instead of vault server

TESTS/SPECS

• Add tests to assert that the service command has been changed to start Vault in agent mode

[jeffbyrnes]

Currently seeing these unit test failures; I’m still getting the hang of Puppet module development (been using Chef until last February):

1) vault on ubuntu-14.04-x86_64  on Debian OS family service with modified options and sysv init contains /etc/init/vault.conf is expected to contain File[/etc/init.d/vault.conf] with content =~ /exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $/
   Failure/Error:
     is_expected.to contain_file('/etc/init.d/vault.conf').
       with_content(%r{exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $})

     expected that the catalogue would contain File[/etc/init.d/vault.conf]
   # ./spec/classes/vault_spec.rb:775:in `block (7 levels) in <top (required)>'

2) vault on debian-9-x86_64  on Debian OS family service with modified options and sysv init contains /etc/init/vault.conf is expected to contain File[/etc/init.d/vault.conf] with content =~ /exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $/
   Failure/Error:
     is_expected.to contain_file('/etc/init.d/vault.conf').
       with_content(%r{exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $})

     expected that the catalogue would contain File[/etc/init.d/vault.conf]
   # ./spec/classes/vault_spec.rb:775:in `block (7 levels) in <top (required)>'

3) vault on debian-8-x86_64  on Debian OS family service with modified options and sysv init contains /etc/init/vault.conf is expected to contain File[/etc/init.d/vault.conf] with content =~ /exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $/
   Failure/Error:
     is_expected.to contain_file('/etc/init.d/vault.conf').
       with_content(%r{exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $})

     expected that the catalogue would contain File[/etc/init.d/vault.conf]
   # ./spec/classes/vault_spec.rb:775:in `block (7 levels) in <top (required)>'

4) vault on ubuntu-16.04-x86_64  on Debian OS family service with modified options and sysv init contains /etc/init/vault.conf is expected to contain File[/etc/init.d/vault.conf] with content =~ /exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $/
   Failure/Error:
     is_expected.to contain_file('/etc/init.d/vault.conf').
       with_content(%r{exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $})

     expected that the catalogue would contain File[/etc/init.d/vault.conf]
   # ./spec/classes/vault_spec.rb:775:in `block (7 levels) in <top (required)>'

5) vault on ubuntu-18.04-x86_64  on Debian OS family service with modified options and sysv init contains /etc/init/vault.conf is expected to contain File[/etc/init.d/vault.conf] with content =~ /exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $/
   Failure/Error:
     is_expected.to contain_file('/etc/init.d/vault.conf').
       with_content(%r{exec start-stop-daemon -u \$USER -g \$GROUP -p \$PID_FILE -x \$VAULT -S -- agent -config=\$CONFIG $})

     expected that the catalogue would contain File[/etc/init.d/vault.conf]
   # ./spec/classes/vault_spec.rb:775:in `block (7 levels) in <top (required)>'

[rgevaert]

Hi @jeffbyrnes . I am in need of this! :+1:

I have fork too with adds some more things, but I would love to include this work! https://github.com/UGent-DICT/puppet-vault

[jeffbyrnes]

@rgevaert cool! I am hoping to work on this some more next week; swamped with some other things at the moment. I’ll take a look at your fork & see if I can synthesize it with this PR.

[rgevaert]

@jeffbyrnes I had a look too, and I hope you don't mind, I have some questions/remarks

On the one hand, I like the approach of using the service_type parameter, but this rules out of running a vault server and a vault agent on the same node. As you can only include this class once (it is a puppet thing).

~Also, this puppet module uses json format for its server config. As far as I can see in the documentation of vault agent it only supports HCL syntax.~

The module has a lot of settings for building config.json but all of them aren't compatible with the vault agent config.

Maybe you have some other work already lined up, but I just wanted to share my comments. Hopefully, I am wrong, am I am saving you from going a dead end.

Also, I found this https://github.com/walkamongus/puppet-vault_agent. This seems what you are looking for too. (And I too :)

[jeffbyrnes]

@rgevaert that is a good point. Running an agent & server on the same node is not something we planned on doing, though obviously you could (provided you customize the ports involved).

And yeah, the config options that the agent does not understand are simply ignored.

You may be on to something with a separate module for the agent…

[jsok]

You may be on to something with a separate module for the agent…

Yes I agree, this would be much more flexible 👍

[jeffbyrnes]

@jsok I still need to circle back on this, and I’d like to cite kyleanderson/consul as an example of a similar Hashi product Puppet module, which covers both the server & agent config.

I’ll come back to this pretty soon; I’m finishing up some sorting some production issues in our Vault setup overall, and then I’ll open up again to clean this up & see it working well.

[TJM]

We also made #191 for this, but apparently this module is "abandoned" ?? Does anyone know if we could get puppet community ( @voxpupuli ) to take over the management of this? or maybe whoever has the best fork should just take it over?

[jeffbyrnes]

@TJM less abandoned, more “stable & not needing much lately”.

That said, that’s now two PRs that provide this functionality. The KyleAnderson/consul module (which is found here on GitHub at https://github.com/solarkennedy/puppet-consul) provides for both server & agent modes, and since they are the same binary, and share some config options, it seems best to provide for either via the same module (contrary to the above conversation).