semaphore
semaphore copied to clipboard
semaphoreui
feat(docker): add py3-netaddr dependency
py3-netaddr is required for some ansible modules such as Zabbix https://github.com/ansible-collections/community.zabbix this PR adds netaddr to the list of dependencies.
The source of the requirement comes from here https://docs.ansible.com/ansible/latest/collections/ansible/utils/docsite/filters_ipaddr.html
fixes #933
Any news on this one? as its currently preventing me from using semaphore with a few of my roles. If it's going to be a while I will look at building my own image
Can I also add my voice behind this, I also am unable to use many of my roles because of this missing dependency.
Greetings.
I don't think hardcoding those dependencies inside the dockerfile is the right solution to this issue as there are many Ansible modules that have additional requirements. Maybe it would be a better solution to allow users to map a 'requirements.txt' file inside the container using a volume and check for its existence/install it at startup.
- AnsibleGuy
How many additional dependencies are there? I don't see an issue in installing a few extra dependencies to ensure users dont have to mess about with the docker image to save a few MB.
I get your point.
These issues would also be related:
- docker => https://github.com/ansible-semaphore/semaphore/issues/1349
- passlib => https://github.com/ansible-semaphore/semaphore/issues/1233
- winrm => https://github.com/ansible-semaphore/semaphore/issues/1175 / https://github.com/ansible-semaphore/semaphore/issues/877
- requests => https://github.com/ansible-semaphore/semaphore/issues/1175
- matrix_client => https://github.com/ansible-semaphore/semaphore/issues/1201
- AnsibleGuy
Awesome. I think it's worth doing to save a bunch of hassle. I will see about adding them into my PR and I will add a docker file into the comments for people to use in the meantime.
A temporary solution to all who need it. Use the below Dockerfile to add py3-jmespath py3-netaddr py3-passlib py3-requests dependencies to semaphore.
FROM semaphoreui/semaphore:latest
USER root
RUN apk add py3-jmespath py3-netaddr py3-passlib py3-requests
USER semaphore
edit: updated version using pip. That adds support for all highlighed packages. Hopfully it helps.
FROM semaphoreui/semaphore:latest
USER root
RUN apk add --no-cache python3 py3-pip
RUN pip install pywinrm jmespath netaddr passlib requests matrix_client
USER semaphore
pywinrm and matrix_client are not available in the Alpine repos. So the next question would be, should any python dependencies be grabbed via pip rather than alpine repos?
I highly recommend at least adding winrm as ansible cant connect to Windows clients without it.
Also to note: Packages installed using PIP are often times newer that those from the package repositories.
Even if all of those ansible.builtin.* and community.* dependencies are hardcoded inside the dockerfile - it would be nice to have a solution for users like me, that are using other Ansible modules (p.e. ansibleguy.opnsense) that have additional requirements
@dgibbs64 Or would you say one should always use a custom dockerfile in that case?
IMHO: In a perfect world, semaphore would react to error messages including 'No Module named' and ask the user if the missing dependency should be installed. (using pip as the module-name could be pulled from the error message and used 1-to-1)
@ansibleguy yeah it would be nice to have a specific feature to install "extra" dependencies, maybe using pip requirements.txt. But that would probably need to be some feature created by the devs of Semaphore.
IMO ansible.builtin and community dependencies should just be hard coded into the default dockerfile and as you suggest any extra dependencies should be added to a custom dockerfile like the example I posted above. If this is the route the dev's choose to take then I think adding some documentation on creating custom dockerfiles would be a good idea.
I have updated my PR to use pip to get all python related dependencies.
As per the discussion, I recommend adding all external dependencies from official and community (e.g community.general) modules by default making it much easier for semaphore users to use the docker image with much fewer complications and disruptions.
Greetings.
I don't think hardcoding those dependencies inside the dockerfile is the right solution to this issue as there are many Ansible modules that have additional requirements. Maybe it would be a better solution to allow users to map a 'requirements.txt' file inside the container using a volume and check for its existence/install it at startup.
- AnsibleGuy
I think this mounting a requirements.txt (also see #1250) is the right way to go here and solves the general case a lot better than hardcoded deps. That PR currently feels a bit simplistic (e.g. no way to update the deps after container startup) but would already make this A LOT nicer.
In the future it might be a good idea to think about a venv per environment/repo/playbook or whatever to which all dependencies, galaxy collections etc are installed would be even better but that is obviously a lot more complex. For now there should be a focus to get #1250 merged.
Hello,
I read this discussion and agree with the overall conclusion to add the possibility to have a custom requirements.txt. But circling back to the orginial report of missing netaddr. If I understand it right the community.* collection is hardcoded and per default included in semaphore. Am I right with this, if so then netaddr has to be too as it is required by community.general.nmcli and here is where my playbooks fail as I used this ansible from the communit.* collection to set ip addresses, dns servers etc. at hosts. And this is at the moment the biggest show stopper for me to use semaphore.
So if semaphore is delivered with a default set of collections I think their python dependencies should be included as well.
Just my 2 Cents on this discussion.
Also: this could be used to update Ansible at 'runtime' (if it is installed using PIP) So related to: https://github.com/ansible-semaphore/semaphore/discussions/1109