creativeprojects
Some suggestions for Ansible
I am aware that the Anisble playbook is a work in progress, so some of this may be planned already, and I am willing to lend a hand with some of these (though I'm new to Anisble, so I may not be the best help)
First of, I see you currently rely on a user defined variable for determining the host os and architecture, and you were looking for a way to auto-detect this. I'm not sure if you're aware, but Ansible actually has a built-in way to do this through the gather facts stage of the playbook: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_vars_facts.html#ansible-facts
The example provided on this page contains the cpu arch and the OS! You can see the CPU arch in both the ansible_architecture and ansible_userspace_architecture keys! You can get the OS in various different forms. You can get (where applicable) the general OS (e.g Linux/Windows) from the ansible_system key, but more specifically you can get everything from the distro family (e.g RedHat/Debian) from ansible_os_family or the specific OS and it's version from ansible_distribution and ansible_lsb with this you could even conditionally install rpm or debian packages using the native package manager or call install scripts dependant on the os. This is a fairly sensible change to make, so I don't see a reason not to use these! You could even detect the presence of systemd or selinux and automatically adjust the default scheduling system.
My next suggestion is a bit more effort and may not be deemed worth it, however I do think it's worth considering, especially if you're thinking of publishing to ansible-galaxy. That is, instead of using a plain playbook, convert your playbook into a role: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_reuse_roles.html This might be something you are already aware of, but it seems logical considering the nature of restic profile. When combined with the above this would potentially allow a user to cleanly integrate restic profile into other playbooks and plays!
As far as I could tell the playbook isn't currently easily accessible from github, but I would be willing to lend a hand with implementing the above. If you want to make a repo, or would rather I make a repo to do this I'd be happy to.
Hey!
Ansible actually has a built-in way to do this through the gather facts stage of the playbook
Nothing is never simple 😢 as it turns out Linux reports arm64 as aarch64, and older arm as armv6l or armv7l and probably even more (these two are from my raspberry pi fleet...). Also, on the example, the architecture is x86_64 whereas my build process is using amd64 (which is more common).
I haven't even tried any *BSD to see what they use 😆
I have been using Ansible for a while, but not very often. I usually setup a playbook when I install a new service on my VPS and I'm done with it for a while 😉
Lately I've updated my playbook to install resticprofile configuration in user directories as well as root. But I'm not very happy with the results, I think we need a configuration object to specify directories, user name, etc.
I have to admit I could certainly do with a bit of help 👍🏻 as usual, time is the only constraint...
Here's my latest version, works on Debian based instances.
Playbook
resticprofile.yml
---
- name: Install restic backup profile
hosts: resticprofile
gather_facts: false
vars_files:
- "../host_vars/{{ vault_vars }}"
vars:
target_bin: /usr/local/bin
temp_dir: /var/tmp/ansible
venv_dir: /var/tmp/venv
tasks:
# Dependencies
- name: Install packages
ansible.builtin.apt:
name:
- python3-venv
- python3-pip
- libssl-dev
state: present
update_cache: true
- name: Install python dependencies
ansible.builtin.pip:
name: github3.py
virtualenv: "{{ venv_dir }}"
virtualenv_command: python3 -m venv
# Gathering facts on restic
- name: Check if restic is installed
ansible.builtin.stat:
path: "{{ target_bin }}/restic"
register: restic_bin
- name: Register restic installation needed
ansible.builtin.set_fact:
install_restic: "{{ not restic_bin.stat.exists }}"
- name: Check restic installed version
ansible.builtin.command: restic version
register: restic_current
when: restic_bin.stat.exists
changed_when: false
- name: Get latest release of restic
vars:
ansible_python_interpreter: '{{ venv_dir }}/bin/python3'
community.general.github_release:
user: restic
repo: restic
action: latest_release
token: "{{ github_token }}"
register: restic_version
- name: Compare restic versions
ansible.builtin.set_fact:
install_restic: "{{ restic_version.tag != restic_current_version }}"
vars:
restic_current_version: "{{ restic_current.stdout | regex_replace('^restic (\\d+\\.\\d+\\.\\d+) .+$', 'v\\1') }}"
when: restic_bin.stat.exists and not ansible_check_mode
# Gathering facts on resticprofile
- name: Check if resticprofile is installed
ansible.builtin.stat:
path: "{{ target_bin }}/resticprofile"
register: resticprofile_bin
- name: Register resticprofile installation needed
ansible.builtin.set_fact:
install_resticprofile: "{{ not resticprofile_bin.stat.exists }}"
- name: Check resticprofile installed version
ansible.builtin.command: resticprofile version
register: resticprofile_current
when: resticprofile_bin.stat.exists
changed_when: false
# older versions of resticprofile need to load a configuration file before executing the version command
failed_when: false
- name: Get latest release of resticprofile
vars:
ansible_python_interpreter: '{{ venv_dir }}/bin/python3'
community.general.github_release:
user: creativeprojects
repo: resticprofile
action: latest_release
token: "{{ github_token }}"
register: resticprofile_version
- name: Compare resticprofile versions
ansible.builtin.set_fact:
install_resticprofile: "{{ resticprofile_version.tag != resticprofile_current_version }}"
vars:
resticprofile_current_version: "{{ resticprofile_current.stdout | regex_replace('^resticprofile version (\\d+\\.\\d+\\.\\d+) .+$', 'v\\1') }}"
when: resticprofile_bin.stat.exists and not ansible_check_mode
# Create an empty temp directory
- name: Remove temp directory
ansible.builtin.file:
path: "{{ temp_dir }}"
state: absent
when: install_restic or install_resticprofile
- name: Create a temp directory if it does not exist
ansible.builtin.file:
path: "{{ temp_dir }}"
state: directory
mode: "0755"
when: install_restic or install_resticprofile
# Install restic
- name: Download restic
ansible.builtin.get_url:
url: "https://github.com/restic/restic/releases/download/{{ restic_version.tag }}/restic_{{ restic_version_number }}_{{ restic_arch }}.bz2"
dest: "{{ temp_dir }}/restic.bz2"
mode: "0640"
vars:
restic_version_number: "{{ restic_version.tag | regex_replace('^v(.*)$', '\\1') }}"
restic_arch: "{{ arch | regex_replace('(^.+_arm)v[67]$', '\\1') }}"
when: install_restic
- name: Extract restic
ansible.builtin.command: "bunzip2 {{ temp_dir }}/restic.bz2"
when: install_restic
changed_when: true
- name: Install restic
ansible.builtin.command: "install {{ temp_dir }}/restic {{ target_bin }}/"
when: install_restic
changed_when: true
# Install resticprofile
- name: Download resticprofile
ansible.builtin.get_url:
url:
"https://github.com/creativeprojects/resticprofile/releases/download\
/{{ resticprofile_version.tag }}/resticprofile_{{ resticprofile_version_number }}_{{ arch }}.tar.gz"
dest: "{{ temp_dir }}/resticprofile.tar.gz"
mode: "0640"
vars:
resticprofile_version_number: "{{ resticprofile_version.tag | regex_replace('^v(.*)$', '\\1') }}"
when: install_resticprofile
- name: Extract resticprofile.tgz
ansible.builtin.unarchive:
src: "{{ temp_dir }}/resticprofile.tar.gz"
dest: "{{ temp_dir }}/"
remote_src: true
when: install_resticprofile
- name: Install resticprofile
ansible.builtin.command: "install {{ temp_dir }}/resticprofile {{ target_bin }}/"
when: install_resticprofile
changed_when: true
- name: Find all users to setup
ansible.builtin.find:
paths: ../resticprofile/{{ inventory_hostname }}
file_type: directory
depth: 1
register: users
delegate_to: localhost
- name: Install resticprofile configuration files
ansible.builtin.include_tasks: ../common_tasks/resticprofile_configuration.yml
vars:
target_user: "{{ files.path | basename }}"
loop: "{{ users.files }}"
loop_control:
loop_var: files
# Cleanup
- name: Remove temp directory
ansible.builtin.file:
path: "{{ temp_dir }}"
state: absent
when: install_restic or install_resticprofile
Task
common_tasks/resticprofile_configuration.yml
# Copy resticprofile configuration files
# vars:
# target_user
---
- name: "Get info on user {{ target_user }}"
ansible.builtin.getent:
database: passwd
key: "{{ target_user }}"
- name: "Get home directory for user {{ target_user }}"
ansible.builtin.set_fact:
user_home: "{{ ansible_facts.getent_passwd[target_user][4] }}"
# TODO: unschedule all profiles (resticprofile unschedule --all)
- name: Ensures resticprofile configuration directory exists
ansible.builtin.file:
path: "{{ user_home }}/resticprofile"
state: directory
owner: "{{ target_user }}"
group: "{{ target_user }}"
mode: "700"
- name: Generates resticprofile configuration file
ansible.builtin.template:
backup: true
src: "{{ item }}"
dest: "{{ user_home }}/resticprofile/"
owner: "{{ target_user }}"
group: "{{ target_user }}"
mode: "0400"
with_fileglob:
- "../resticprofile/{{ inventory_hostname }}/{{ target_user }}/profiles.*"
- name: Install other resticprofile files (like excludes)
ansible.builtin.template:
src: "{{ item }}"
dest: "{{ user_home }}/resticprofile/"
owner: "{{ target_user }}"
group: "{{ target_user }}"
mode: "0444"
with_fileglob:
- "../resticprofile/{{ inventory_hostname }}/{{ target_user }}/copy/*"
- name: Install keys
ansible.builtin.copy:
src: "{{ item }}"
dest: "{{ user_home }}/resticprofile/"
decrypt: true
owner: "{{ target_user }}"
group: "{{ target_user }}"
mode: "0400"
with_fileglob:
- "../resticprofile/{{ inventory_hostname }}/{{ target_user }}/keys/*"
# TODO: schedule all profiles (resticprofile schedule --all)
Files
For this configuration to work, you need to create directories and files this way:
resticprofile
- vps01
- root
- copy
* excludes
- keys
* azure-key
* profiles.yml
- user
- keys
* s3-key
* profiles.conf
First level is machine name, second level is username, then the files to copy (copy for normal files & keys for encrypted files)
You get the idea 😉
I think a configuration object would be better with the list of files to copy, where to put them, their owner, etc.
Thanks for the fast response!
I feel like you could potentially account for that by using where statements to check for certain values and then work from there! Seen as ansible roles can include custom code (e.g python scripts) you could also run different python or shell scripts depending on the output of that var. I think first and foremost we should aim to target popular OSes and distros (e.g Windows, Mac and Linux) we can always work to implement BSDs or other nixes should they be requested (in which case we'd probably also have someone who uses those operating systems to test)
Haha Ansible does seem to be good for that sort of thing. I'm very new too it myself too (I have been using it for a week, so forgive me if I'm not familiar with certain aspects)
I think that's where a role would come in useful. You can set up defaults and variables to be used. That said you could also let the user decide which user to use (that was a sentence and a half) using the become option and writing it to something like ~/.config I feel like (by default at least)
Sorry I wrote and sent that message just before that new comment popped up. That is a long playbook so I'll need some time to read through it. I do think some form of config object would be helpful! I'll give it a look over when I get chance and see if I can make any improvements worthy of being shared! I might throw it in a repo too, just to have some proper version control on it to track my changes. Would you be interested in creating one, or would you rather I did that on my own for the time being?
I'm perfectly fine if you do a repo on your own 👍🏻
I did use this playbook to update Restic to 0.18.0 on all my machines this morning, it saves me so much time!
Thank you for your help 😉
No problem! I'm a fedora user myself! (I do have a couple of debian/ubuntu systems though) As such it's in my interest to create something that's more flexible to different distros! I've been looking to simplify the configuration of restic across my various systems for a while now, and I think combined with anisble, resticprofile looks very promising to fill that hole.
Hi @creativeprojects @Cwavs
I'm looking to potentially move from autorestic to resticprofile. I already maintain an Ansible role for autorestic. Maybe we could create a similar role for resticprofile.
I had a look, and your ansible role for autorestic looks pretty cool indeed 👍🏻
I just did a quick playbook, but it's too specific to my needs (https://creativeprojects.github.io/resticprofile/installation/ansible/index.html) I use it regularly to upgrade restic/resticprofile on all my servers though
So, please go ahead, I'd love to try it 😎
The thing that is missing in my playbook is rescheduling the tasks.
This isn't implemented yet because with previous versions of resticprofile, you couldn't delete a schedule by removing a profile.
Since version 0.30.0 we can safely run resticprofile unschedule --all and nothing will be left behind 😉