ansible feat: Add prometheus prometheus_provisioning

Add support for prometheus_provisioning_synced (fixes https://github.com/prometheus-community/ansible/issues/147).

Nov 06 '24 11:11 sergiocharpineljr

Docs Build 📝

Thank you for contribution!✨

The docs for this PR have been published here: https://prometheus-community.github.io/ansible/pr/453

You can compare to the docs for the main branch here: https://prometheus-community.github.io/ansible/branch/main

The docsite for this PR is also available for download as an artifact from this run: https://github.com/prometheus-community/ansible/actions/runs/11702477098

File changes:

M prometheus_role.html

Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded. See the file list and check the published docs to see those files.

diff --git a/home/runner/work/ansible/ansible/docsbuild/base/prometheus_role.html b/home/runner/work/ansible/ansible/docsbuild/head/prometheus_role.html
index da4a0c4..9cb4b55 100644
--- a/home/runner/work/ansible/ansible/docsbuild/base/prometheus_role.html
+++ b/home/runner/work/ansible/ansible/docsbuild/head/prometheus_role.html
@@ -312,27 +312,39 @@ To check whether it is installed, run <code class="code docutils literal notrans
 </div></td>
 </tr>
 <tr class="row-even"><td><div class="ansible-option-cell">
+<div class="ansibleOptionAnchor" id="parameter-main--prometheus_provisioning_synced"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-provisioning-synced"><strong>prometheus_provisioning_synced</strong></p>
+<a class="ansibleOptionLink" href="#parameter-main--prometheus_provisioning_synced" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">boolean</span></p>
+</div></td>
+<td><div class="ansible-option-cell"><p>Should the provisioning of alert rule files be kept synced. If true, previous provisioned files will be removed if not referenced anymore.</p>
+<p class="ansible-option-line"><strong class="ansible-option-choices">Choices:</strong></p>
+<ul class="simple">
+<li><p><code class="ansible-option-default-bold docutils literal notranslate"><strong><span class="pre">false</span></strong></code> <span class="ansible-option-choices-default-mark">← (default)</span></p></li>
+<li><p><code class="ansible-option-choices-entry docutils literal notranslate"><span class="pre">true</span></code></p></li>
+</ul>
+</div></td>
+</tr>
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_read_only_dirs"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-read-only-dirs"><strong>prometheus_read_only_dirs</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_read_only_dirs" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Additional paths that Prometheus is allowed to read (useful for SSL certs outside of the config directory)</p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_remote_read"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-remote-read"><strong>prometheus_remote_read</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_remote_read" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Remote read. It is compatible with the <a class="reference external" href="https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_read">official configuration</a></p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_remote_write"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-remote-write"><strong>prometheus_remote_write</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_remote_write" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Remote write. Compatible with the <a class="reference external" href="https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write">official configuration</a></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_scrape_config_files"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-scrape-config-files"><strong>prometheus_scrape_config_files</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_scrape_config_files" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
@@ -341,7 +353,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">[&quot;prometheus/scrape_configs/*.yml&quot;,</span> <span class="pre">&quot;prometheus/scrape_configs/*.json&quot;]</span></code></p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_scrape_configs"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-scrape-configs"><strong>prometheus_scrape_configs</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_scrape_configs" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=dictionary</span></p>
 </div></td>
@@ -349,7 +361,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">[{&quot;job_name&quot;:</span> <span class="pre">&quot;prometheus&quot;,</span> <span class="pre">&quot;metrics_path&quot;:</span> <span class="pre">&quot;{{</span> <span class="pre">prometheus_metrics_path</span> <span class="pre">}}&quot;,</span> <span class="pre">&quot;static_configs&quot;:</span> <span class="pre">[{&quot;targets&quot;:</span> <span class="pre">[&quot;{{</span> <span class="pre">ansible_fqdn</span> <span class="pre">|</span> <span class="pre">default(ansible_host)</span> <span class="pre">|</span> <span class="pre">default('localhost')</span> <span class="pre">}}:9090&quot;]}]},</span> <span class="pre">{&quot;file_sd_configs&quot;:</span> <span class="pre">[{&quot;files&quot;:</span> <span class="pre">[&quot;{{</span> <span class="pre">prometheus_config_dir</span> <span class="pre">}}/file_sd/node.yml&quot;]}],</span> <span class="pre">&quot;job_name&quot;:</span> <span class="pre">&quot;node&quot;}]</span></code></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_static_targets_files"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-static-targets-files"><strong>prometheus_static_targets_files</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_static_targets_files" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">list</span> / <span class="ansible-option-elements">elements=string</span></p>
 </div></td>
@@ -357,7 +369,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">[&quot;prometheus/targets/*.yml&quot;,</span> <span class="pre">&quot;prometheus/targets/*.json&quot;]</span></code></p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_stop_timeout"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-stop-timeout"><strong>prometheus_stop_timeout</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_stop_timeout" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -365,7 +377,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;600s&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_storage_retention"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-storage-retention"><strong>prometheus_storage_retention</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_storage_retention" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -373,7 +385,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;30d&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_storage_retention_size"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-storage-retention-size"><strong>prometheus_storage_retention_size</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_storage_retention_size" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -383,7 +395,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;0&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_system_group"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-system-group"><strong>prometheus_system_group</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_system_group" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -391,7 +403,7 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;prometheus&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_system_user"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-system-user"><strong>prometheus_system_user</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_system_user" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -399,14 +411,14 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;prometheus&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_targets"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-targets"><strong>prometheus_targets</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_targets" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">dictionary</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Targets which will be scraped.</p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_version"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-version"><strong>prometheus_version</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_version" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
@@ -415,21 +427,21 @@ To check whether it is installed, run <code class="code docutils literal notrans
 <p class="ansible-option-line"><strong class="ansible-option-default-bold">Default:</strong> <code class="ansible-option-default docutils literal notranslate"><span class="pre">&quot;2.54.1&quot;</span></code></p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_web_config"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-web-config"><strong>prometheus_web_config</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_web_config" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">dictionary</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>A Prometheus <a class="reference external" href="https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md">web config yaml</a> for configuring TLS and auth.</p>
 </div></td>
 </tr>
-<tr class="row-even"><td><div class="ansible-option-cell">
+<tr class="row-odd"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_web_external_url"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-web-external-url"><strong>prometheus_web_external_url</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_web_external_url" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>External address on which prometheus is available. Useful when behind reverse proxy. Ex. `http://example.org/prometheus`</p>
 </div></td>
 </tr>
-<tr class="row-odd"><td><div class="ansible-option-cell">
+<tr class="row-even"><td><div class="ansible-option-cell">
 <div class="ansibleOptionAnchor" id="parameter-main--prometheus_web_listen_address"></div><p class="ansible-option-title" id="ansible-collections-prometheus-prometheus-prometheus-role-parameter-main-prometheus-web-listen-address"><strong>prometheus_web_listen_address</strong></p>
 <a class="ansibleOptionLink" href="#parameter-main--prometheus_web_listen_address" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">string</span></p>
 </div></td>

Nov 06 '24 11:11 github-actions[bot]

I tested this locally but didn't include any tests. If you feel like we should, please let me know and give me some guidance on the best approach to follow.

Nov 06 '24 11:11 sergiocharpineljr

Thanks. I wonder if it might be cleaner to simply switch the Copy custom alerting rule files task over to the ansible.posix.synchronize module with delete: true to solve this. Also generally not a fan of using the json_query filter unless absolutely necessary.

I've followed the suggestion in https://github.com/prometheus-community/ansible/issues/147 to base this on Grafana's code: https://github.com/grafana/grafana-ansible-collection/blob/84d6af3b1c3991d6c560a8fe9c09d8fc1811f520/roles/grafana/tasks/dashboards.yml#L136-L142

The ansible.posix.synchronize might be tricky because of the ansible_managed.yml file we have in the same dir. So not sure how to make this idempotent.

json_query was used because of the default filter. Doing __rules_managed_copied.dest directly causes issues when prometheus_alert_rules = [] and the __rules_managed_copied var is not created.

Nov 06 '24 14:11 sergiocharpineljr

hi @sergiocharpineljr

ansible.posix.synchronize can work as well.

FYI

---
- name: Synchronize directories using ansible.posix.synchronize
  hosts: localhost
  tasks:
    - name: Synchronize foo/rules to bar/rules with delete option
      ansible.posix.synchronize:
        src: ~/tmp/foo/rules/
        dest: ~/tmp/bar/rules/
        delete: yes
        recursive: yes
        rsync_opts:
          - "--exclude='ansible_managed.yml'"

jack@v:~/tmp$ ansible-playbook a.yaml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Synchronize directories using ansible.posix.synchronize] **********************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Synchronize foo/rules to bar/rules with delete option] ************************************************************************************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

jack@v:~/tmp$ tree foo/rules/ bar/rules/
foo/rules/
└── aaa.yml
bar/rules/
├── aaa.yml
└── ansible_managed.yml

2 directories, 3 files

Nov 08 '24 14:11 zhan9san

@zhan9san @gardar Ok, I can try the synchronize approach. Should we leave it as optional, keeping the current method as default (or setting delete: false) or do you think we should always force copy instead?

Nov 08 '24 16:11 sergiocharpineljr

I prefer using synchronize with delete: true as default.

@gardar Your thought?

Nov 10 '24 16:11 zhan9san

I prefer using synchronize with delete: true as default.

@gardar Your thought?

I agree; ideally, the roles should strictly manage how Prometheus is configured. However, I understand that’s not always the case, so having it as an optional feature could be useful.

Another concern I have is that using the synchronize module adds rsync as a dependency to the role, and rsync isn’t always installed by default.

Nov 13 '24 00:11 gardar

I prefer using synchronize with delete: true as default. @gardar Your thought?

I agree; ideally, the roles should strictly manage how Prometheus is configured. However, I understand that’s not always the case, so having it as an optional feature could be useful.

Another concern I have is that using the synchronize module adds rsync as a dependency to the role, and rsync isn’t always installed by default.

Yes, well remembered. So @zhan9san @gardar can we stick with the original proposal?

Dec 13 '24 20:12 sergiocharpineljr

feat: Add prometheus prometheus_provisioning_synced support

Docs Build 📝