Add consul_policy, consul_role and consul_token modules

[Hakon]

• add consul_policy, consul_role and consul_token modules refactor consul_session to not use the consul python package
• Add some missing documentation, remove python-consul and pyhcl requirements

SUMMARY

The consul_acl creates the old type of tokens (consul < 1.4.0) which is deprecated and removed from consul 1.11.0. The new consul_policy, consul_role and consul_token modules is compatible with the new ACL system. The consul_acl module is kept for backwards compatibility.

The consul_token module could be merged into consul_acl, but it could possibly create backward compatibility issues with existing uses of consul_acl.

I have avoided using any "consul client" python package and integrate with the consul API directly. This is a tradeoff between maintaining our own client and having 3rd party package dependencies. The consul HTTP is quite stable so I think this is an OK tradeoff to make.

I also refactored the consul_session module to use requests and the HTTP API directly because of a bug where the 3rd party consul client used did not support authentication with tokens. Why this has not been reported before is beyond me, but there is a bug on the python package issue tracker.

Note that I have not implemented a consul_role module but this should be a relatively simple task for someone who wants to get into ansible module development.

Fixes #1749

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

consul_policy consul_role consul_token

[ansibullbot]

cc @sgargan click here for bot help

[github-actions[bot]]

Docs Build 📝

Thank you for contribution!✨

The docsite for this PR is available for download as an artifact from this run: https://github.com/ansible-collections/community.general/actions/runs/3152857778

File changes:

• A collections/community/general/consul_policy_module.html
• A collections/community/general/consul_role_module.html
• A collections/community/general/consul_token_module.html
• M collections/community/general/consul_kv_module.html
• M collections/community/general/consul_session_module.html
• M collections/community/general/copr_module.html
• M collections/community/general/index.html
• M collections/index_module.html

Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded. See the file list and check the published docs to see those files.

diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/consul_kv_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/consul_kv_module.html
index 2104848..cb67ff3 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/consul_kv_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/consul_kv_module.html
@@ -22,7 +22,7 @@
         <script src="../../../_static/sphinx_highlight.js"></script>
     <script src="../../../_static/js/theme.js"></script>
     <link rel="search" title="Search" href="../../../search.html" />
-    <link rel="next" title="community.general.consul_session module – Manipulate consul sessions" href="consul_session_module.html" />
+    <link rel="next" title="community.general.consul_policy module – Manipulate Consul policies" href="consul_policy_module.html" />
     <link rel="prev" title="community.general.consul_acl module – Manipulate Consul ACL keys and rules" href="consul_acl_module.html" /><!-- extra head elements for Ansible beyond RTD Sphinx Theme -->
 
 
@@ -350,7 +350,7 @@ see <a class="reference internal" href="#ansible-collections-community-general-c
 
 <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
         <a href="consul_acl_module.html" class="btn btn-neutral float-left" title="community.general.consul_acl module – Manipulate Consul ACL keys and rules" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
-        <a href="consul_session_module.html" class="btn btn-neutral float-right" title="community.general.consul_session module – Manipulate consul sessions" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+        <a href="consul_policy_module.html" class="btn btn-neutral float-right" title="community.general.consul_policy module – Manipulate Consul policies" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
     </div>
 
   <hr/>
diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/consul_session_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/consul_session_module.html
index 4236a35..3c89f0b 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/consul_session_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/consul_session_module.html
@@ -22,8 +22,8 @@
         <script src="../../../_static/sphinx_highlight.js"></script>
     <script src="../../../_static/js/theme.js"></script>
     <link rel="search" title="Search" href="../../../search.html" />
-    <link rel="next" title="community.general.copr module – Manage one of the Copr repositories" href="copr_module.html" />
-    <link rel="prev" title="community.general.consul_kv module – Manipulate entries in the key/value store of a consul cluster" href="consul_kv_module.html" /><!-- extra head elements for Ansible beyond RTD Sphinx Theme -->
+    <link rel="next" title="community.general.consul_token module – Manipulate Consul tokens" href="consul_token_module.html" />
+    <link rel="prev" title="community.general.consul_role module – Manipulate Consul roles" href="consul_role_module.html" /><!-- extra head elements for Ansible beyond RTD Sphinx Theme -->
 
 
 
@@ -332,6 +332,7 @@ see <a class="reference internal" href="#ansible-collections-community-general-c
 <h3>Authors<a class="headerlink" href="#authors" title="Permalink to this heading"></a></h3>
 <ul class="simple">
 <li><p>Steve Gargan (&#64;sgargan)</p></li>
+<li><p>Håkon Lerring (&#64;Hakon)</p></li>
 </ul>
 </section>
 <section id="collection-links">
@@ -352,8 +353,8 @@ see <a class="reference internal" href="#ansible-collections-community-general-c
           
 
 <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
-        <a href="consul_kv_module.html" class="btn btn-neutral float-left" title="community.general.consul_kv module – Manipulate entries in the key/value store of a consul cluster" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
-        <a href="copr_module.html" class="btn btn-neutral float-right" title="community.general.copr module – Manage one of the Copr repositories" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+        <a href="consul_role_module.html" class="btn btn-neutral float-left" title="community.general.consul_role module – Manipulate Consul roles" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+        <a href="consul_token_module.html" class="btn btn-neutral float-right" title="community.general.consul_token module – Manipulate Consul tokens" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
     </div>
 
   <hr/>
diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/copr_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/copr_module.html
index 6a12ddc..4d47ea7 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/copr_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/copr_module.html
@@ -23,7 +23,7 @@
     <script src="../../../_static/js/theme.js"></script>
     <link rel="search" title="Search" href="../../../search.html" />
     <link rel="next" title="community.general.cpanm module – Manages Perl library dependencies." href="cpanm_module.html" />
-    <link rel="prev" title="community.general.consul_session module – Manipulate consul sessions" href="consul_session_module.html" /><!-- extra head elements for Ansible beyond RTD Sphinx Theme -->
+    <link rel="prev" title="community.general.consul_token module – Manipulate Consul tokens" href="consul_token_module.html" /><!-- extra head elements for Ansible beyond RTD Sphinx Theme -->
 
 
 
@@ -299,7 +299,7 @@ see <a class="reference internal" href="#ansible-collections-community-general-c
           
 
 <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
-        <a href="consul_session_module.html" class="btn btn-neutral float-left" title="community.general.consul_session module – Manipulate consul sessions" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+        <a href="consul_token_module.html" class="btn btn-neutral float-left" title="community.general.consul_token module – Manipulate Consul tokens" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
         <a href="cpanm_module.html" class="btn btn-neutral float-right" title="community.general.cpanm module – Manages Perl library dependencies." accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
     </div>
 
diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/index.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/index.html
index f9c6af5..85128fe 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/index.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/index.html
@@ -233,7 +233,10 @@
 <li><p><a class="reference internal" href="consul_module.html#ansible-collections-community-general-consul-module"><span class="std std-ref">consul module</span></a> – Add, modify &amp; delete services within a consul cluster.</p></li>
 <li><p><a class="reference internal" href="consul_acl_module.html#ansible-collections-community-general-consul-acl-module"><span class="std std-ref">consul_acl module</span></a> – Manipulate Consul ACL keys and rules</p></li>
 <li><p><a class="reference internal" href="consul_kv_module.html#ansible-collections-community-general-consul-kv-module"><span class="std std-ref">consul_kv module</span></a> – Manipulate entries in the key/value store of a consul cluster</p></li>
+<li><p><a class="reference internal" href="consul_policy_module.html#ansible-collections-community-general-consul-policy-module"><span class="std std-ref">consul_policy module</span></a> – Manipulate Consul policies</p></li>
+<li><p><a class="reference internal" href="consul_role_module.html#ansible-collections-community-general-consul-role-module"><span class="std std-ref">consul_role module</span></a> – Manipulate Consul roles</p></li>
 <li><p><a class="reference internal" href="consul_session_module.html#ansible-collections-community-general-consul-session-module"><span class="std std-ref">consul_session module</span></a> – Manipulate consul sessions</p></li>
+<li><p><a class="reference internal" href="consul_token_module.html#ansible-collections-community-general-consul-token-module"><span class="std std-ref">consul_token module</span></a> – Manipulate Consul tokens</p></li>
 <li><p><a class="reference internal" href="copr_module.html#ansible-collections-community-general-copr-module"><span class="std std-ref">copr module</span></a> – Manage one of the Copr repositories</p></li>
 <li><p><a class="reference internal" href="cpanm_module.html#ansible-collections-community-general-cpanm-module"><span class="std std-ref">cpanm module</span></a> – Manages Perl library dependencies.</p></li>
 <li><p><a class="reference internal" href="cronvar_module.html#ansible-collections-community-general-cronvar-module"><span class="std std-ref">cronvar module</span></a> – Manage variables in crontabs</p></li>
diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/index_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/index_module.html
index 8afcaf2..428bbbd 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/index_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/index_module.html
@@ -177,7 +177,10 @@
 <li><p><a class="reference internal" href="community/general/consul_module.html#ansible-collections-community-general-consul-module"><span class="std std-ref">community.general.consul</span></a> – Add, modify &amp; delete services within a consul cluster.</p></li>
 <li><p><a class="reference internal" href="community/general/consul_acl_module.html#ansible-collections-community-general-consul-acl-module"><span class="std std-ref">community.general.consul_acl</span></a> – Manipulate Consul ACL keys and rules</p></li>
 <li><p><a class="reference internal" href="community/general/consul_kv_module.html#ansible-collections-community-general-consul-kv-module"><span class="std std-ref">community.general.consul_kv</span></a> – Manipulate entries in the key/value store of a consul cluster</p></li>
+<li><p><a class="reference internal" href="community/general/consul_policy_module.html#ansible-collections-community-general-consul-policy-module"><span class="std std-ref">community.general.consul_policy</span></a> – Manipulate Consul policies</p></li>
+<li><p><a class="reference internal" href="community/general/consul_role_module.html#ansible-collections-community-general-consul-role-module"><span class="std std-ref">community.general.consul_role</span></a> – Manipulate Consul roles</p></li>
 <li><p><a class="reference internal" href="community/general/consul_session_module.html#ansible-collections-community-general-consul-session-module"><span class="std std-ref">community.general.consul_session</span></a> – Manipulate consul sessions</p></li>
+<li><p><a class="reference internal" href="community/general/consul_token_module.html#ansible-collections-community-general-consul-token-module"><span class="std std-ref">community.general.consul_token</span></a> – Manipulate Consul tokens</p></li>
 <li><p><a class="reference internal" href="community/general/copr_module.html#ansible-collections-community-general-copr-module"><span class="std std-ref">community.general.copr</span></a> – Manage one of the Copr repositories</p></li>
 <li><p><a class="reference internal" href="community/general/cpanm_module.html#ansible-collections-community-general-cpanm-module"><span class="std std-ref">community.general.cpanm</span></a> – Manages Perl library dependencies.</p></li>
 <li><p><a class="reference internal" href="community/general/cronvar_module.html#ansible-collections-community-general-cronvar-module"><span class="std std-ref">community.general.cronvar</span></a> – Manage variables in crontabs</p></li>

[Hakon]

/rebuild_failed

[Hakon]

ready_for_review

[Hakon]

Please note that we recommend to not add multiple new modules in one PR, but to instead start with one PR for one new module, get that polished and merged, and then create new PRs for the other modules (and already incorporate all the comments from the first PR). (Generally also see CONTRIBUTING.md.)

Ah, thanks! I thought I had read the CONTRIBUTING.md, but apparently not thoroughly enough. Do you want me to split this into 3 PRs now?

[felixfontein]

You can also keep the two other modules in here, but the PR will likely take a lot longer to get reviewed and merged (large PRs tend to scare reviewers off :) ).

[felixfontein]

Please note that in #5461 the collection repository was restructured to remove the directory tree in plugins/modules/, and the corresponding tree in tests/unit/plugins/modules/. Your PR adds new files into this hierarchy. Please rebase with the current main branch and move your files directly into plugins/modules/. You also can remove the changes to meta/runtime.yml, these are not needed anymore (the corresponding section was removed from CONTRIBUTING.md), and make sure to adjust the new entries in .github/BOTMETA.yml as well.