Execute task-template within python-virtual environment or container

[ccuz]

The task-template are run by https://github.com/ansible-semaphore/semaphore/blob/develop/services/tasks/runner.go that calls https://github.com/ansible-semaphore/semaphore/blob/develop/lib/AnsiblePlaybook.go which starts the ansible-playbook command into a sub-process.

We have some shortcomings with this simple effective execution:

• Using '-vvv' while execution the ansible script displays the environment variable that started semaphore (i.e. db password, token, ... if configured through the semaphore-wrapper using environment variable).
• Per project, we use specific version of ansible and collections (using roles/requirements.yaml and collections/requirements.yaml). That works, though we also patches these collections using git-patches.

One idea, would be always build a python venv (https://docs.python.org/3/library/venv.html) before running the task-template process, thus dependencies doesn't clashes.

Another would be to create a container (i.e. docker image) an run the ansible within the provided container image. This provides a solution for problem 2 and ensure no semaphore environment variable leeks into the ansible-playbook execution. Not sure if possible in go while keeping everything simple, maybe using https://www.redhat.com/sysadmin/podman-inside-container. This would match https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.1/html-single/red_hat_ansible_automation_platform_upgrade_and_migration_guide/index#upgrading-to-ees

Maybe, the more flexible would be to allow a '/etc/semaphore/playground-setup.sh' to be run just before the ansible-playbook command, controlled by semaphore admin. Thus one could spin a container and pass the 'ansible-playbook' command and argument to it or setup a python venv before each execution. Or just use git hooks https://schacon.github.io/git/githooks.html#_post_checkout to perform environment setup (i.e. patching collections or setting up python venv).

[ccuz]

I ended up creating a hooks/post-checkout and hooks/post-merge within the ansible git project:

#!/bin/bash
echo "Git repo hook: $0"

echo "Create a python venv"
python3 -m .venv myproject
source .venv/bin/activate
pip install -r pip-packages.txt

echo "Installing roles/requirements.yml"
ansible-galaxy role install -r roles/requirements.yml --force

echo "Installing collections/requirements.yml"
ansible-galaxy collection install -r collections/requirements.yml --force

echo "Patching buggy collections"
git apply $(ls patches/*.patch)

In the dockerfile where semaphore is installed, I added: RUN git config --global core.hooksPath hooks Thus any ansible project getting checked-out by semaphore has the possibility to correctly setup the requirements and have clean dependencies not conflicting with other projects.

But the actual 'ansible-playbook' still doesn't run inside the virtual environment. It would need to run 'source myproject/bin/activate' just before actually running ansible-playbook.

[ccuz]

For the container runtime integration with podman:

• Without podman local socket: https://github.com/containers/podman/discussions/9076
• Podman RestAPI (with local socket to podman deamon, requires podman to be installed on the system): https://docs.podman.io/en/latest/_static/api.html and howto start a container from golang: https://github.com/containers/podman/tree/main/pkg/bindings (the original blog post: https://podman.io/blogs/2020/08/10/podman-go-bindings.html#create-start-container)