Ansible Automation Platform "Missing required library \"pan-os-python\"." error

[david-johnstone]

Describe the bug

Currently running Ansible Automation Platform AAP 2.2 which uses podman containers to execute templates and still receiving the same error as https://github.com/PaloAltoNetworks/pan-os-ansible/issues/324 despite having pan-os-python being installed in the container. Using latest collection from Galaxy paloaltonetworks-panos-3.0.0

Requesting reopening: https://github.com/PaloAltoNetworks/pan-os-ansible/issues/324

Error:

{ "pypi": "https://pypi.org/project/pan-python", "syspath": [ "/tmp/ansible_panos_object_facts_payload_164rchcs/ansible_panos_object_facts_payload.zip", "/usr/lib64/python36.zip", "/usr/lib64/python3.6", "/usr/lib64/python3.6/lib-dynload", "/usr/lib64/python3.6/site-packages", "/usr/lib/python3.6/site-packages" ], "msg": "Missing required library "pan-python".",

Expected behavior

Library should be identified as being installed when calling panos_object_facts

Current behavior

When calling the panos_object_facts function from paloaltonetworks.panos, the collection doesn't identify pan-python despite it being installed in the container.

Possible solution

Unsure.

Steps to reproduce

Files required to create Podman container:

bindep.txt

gcc python3-libselinux [platform:redhat] python3 [platform:dpkg]

execution-environment.yaml

version: 2 dependencies: galaxy: requirements.yml python: requirements.txt system: bindep.txt

additional_build_steps: prepend: | RUN pip3 install --upgrade pip setuptools append:

• RUN ls -la /etc

requirements.txt

psutil pandevice==0.14.0 certifi chardet==3.0.4 idna==2.8 pan-os-python pan-python requests==2.22.0 urllib3==1.25.11 xmltodict==0.12.0 openpyxl selinux

requirements.yml

collections:

paloaltonetworks.panos community.general Build command:

ansible-builder build --tag /ee_panorama_1_03 -v3

Error from Ansible Automation Platform Console: { "pypi": "https://pypi.org/project/pan-python", "syspath": [ "/tmp/ansible_panos_object_facts_payload_164rchcs/ansible_panos_object_facts_payload.zip", "/usr/lib64/python36.zip", "/usr/lib64/python3.6", "/usr/lib64/python3.6/lib-dynload", "/usr/lib64/python3.6/site-packages", "/usr/lib/python3.6/site-packages" ], "msg": "Missing required library "pan-python".",

Container pan-os-python Details

Downloading https://galaxy.ansible.com/download/paloaltonetworks-panos-3.0.0.tar.gz to /home/runner/.ansible/tmp/ansible-local-1v3483z7d/tmpwwp4fwrg/paloaltonetworks-panos-3.0.0-xk3y1ej6 Installing 'paloaltonetworks.panos:3.0.0' to '/usr/share/ansible/collections/ansible_collections/paloaltonetworks/panos' paloaltonetworks.panos:3.0.0 was installed successfully

Task Object panos_object_facts

Any assistance would be greatly appreciated as we are currently migrating from Tower to AAP.

Screenshots

Context

Your Environment

• Collection: paloaltonetworks.panos
• Python: 3.6
• Ansible: Ansible Automation Platform 2.2 ansible>=2.12
• Latest versions used.

[welcome-to-palo-alto-networks[bot]]

:tada: Thanks for opening your first issue here! Welcome to the community!

[GauthierSgds]

Hello,

Same issue here, any update?

Regards

[david-johnstone]

None yet. We have a piece of automation that has stalled as a result of this bug.

[shinmog]

There are two python libraries, both are required for the Ansible collection:

• pan-python
• pan-os-python

This issue's title is saying you're missing "pan-os-python", but the error message is actually saying "pan-python". So there's definitely some confusion here.

#324 was closed out because Ansible sometimes likes to look in places users don't expect, and python as a runtime environment is wonky as all get-out sometimes. In your error, due to the enhancements from #324, we know what the paths are that Ansible is looking in for both pan-python and pan-os-python. At the very least, pan-python is not in any of the paths mentioned in syspath in the error output.

Make sure you've installed both pan-python and pan-os-python in one of the paths mentioned in your syspath, probably "/usr/lib64/python3.6/site-packages" based on what I see.

One final thing to note: Ansible is moving towards a minimum python version of 3.8. We've had to update our CI runtime environments because some other the newer Ansible runtimes bomb out if you're not running 3.8. I would recommend to start moving in that direction when you finish dealing with the library location errors.

[Dixie9000]

@shinmog Also running AAP 2.2.

I am having the exact same issue. But I'm running Python 3.8 and have all the required python modules installed. I don't even get proper error code now when invoking panos_op. I just get "Missing required libraries." I have dug through controller logs and the API and debug level 3 to try to get more information but I get nothing. I verified that all the packages are installed in the various syspath folders of Python3.8 in my execution environment container. I have racked my brains trying to get it to work, but I'm completely confused at this point as to why it isn't working.

Update: You can get around this bug by using a URI module to run the commands you want, like so:

- name: Gather system info
      uri:
        validate_certs: no
        url: 'https://{{ ansible_host }}/api/?type=op&cmd=<show><system><info></info></system></show>&key={{ api_key }}'
        return_content: yes
      register: res1
      ignore_errors: yes

[shinmog]

@Dixie9000

The generic error of "Missing required libraries" only happens when you're missing a module specific library. For panos_op, that library is xmltodict. Unfortunately, module specific python library dependencies are checked before the generic ones (pan-os-python and pan-python), so you don't see what the python paths are that Ansible is looking for.

I'd recommend using a module different from panos_op, see if you get a library error or not, and if you do, you'll have the paths where Ansible is checking to ensure that dependencies are installed where Ansible is looking.

[shinmog]

@Dixie9000

As an example, panos_address_object or something (gathered_filter was added in v2.11.0):

- name: 'Get foo objects'
  panos_address_object:
    provider: '{{ provider_dict }}'
    state: 'gathered'
    gathered_filter: 'name contains "foo"'

[Dixie9000]

@shinmog So, I appreciate your help, but I think it's possible some type of RHEL8 update broke the paloaltonetworks.panos module on AAP 2.2.

I got the paths for you from a simple module just panos_facts, which worked a few months ago without issue with no configuration changes. It all of the sudden stopped working in the last week or so, which just so happens to be after the patching window.

{
"pypi": "https://pypi.org/project/pan-python",
  "syspath": [
    "/tmp/ansible_panos_facts_payload_iekbe07f/ansible_panos_facts_payload.zip",
    "/usr/lib64/python36.zip",
    "/usr/lib64/python3.6",
    "/usr/lib64/python3.6/lib-dynload",
    "/usr/lib64/python3.6/site-packages",
    "/usr/lib/python3.6/site-packages"
  ],
  "msg": "Missing required library \"pan-python\".",

As you can see here, for some reason it's looking in Python 3.6, I have no idea why. It worked perfectly fine a month ago. I didn't set it to do that anywhere in the container. The syspath of Python on my execution environment contains absolutely no 3.6 references, it's all pointing to a newer version. As you can see from the screenshot I posted above. I even built another container to test in and directly installed the panos collection into the container instead of calling it from the project root in collections/requirements.yml. Neither resolved it.

I have been troubleshooting this for 3 days and cannot get it resolved. I have talked to other people who have no issues on regular community CLI ansible with the panos module, it's only an AAP problem.

I have reached out to a Redhat Architect about this, contacted the devs here, and reached out to RedHat support and still do not have a resolution.

[Dixie9000]

So the only fix I've found, is to set the interpreter manually at the template or inventory level:

---
ansible_python_interpreter: /usr/bin/python3

For awareness for anyone else. I also tried adding this to the ansible.cfg for the container build file:

[defaults]
interpreter_python=/usr/bin/python3

I couldn't get the ansible.cfg file to circumvent 3.6 for whatever reason, even after specifying it in the ansible.cfg, so YMMV, but the top yaml variable works in getting around this issue.

[RyanMBess]

that ansible.cfg file must be in the root of your github repo.

[Dixie9000]

Indeed, I tested that and it's working.

You could also take that one-liner ansible.cfg and copy it into the /etc/ansible/ansible.cfg directory on your container, and it would probably also work, but I haven't tested that yet.

[shinmog]

@david-johnstone

Since you're running AAP 2.2 the same as @Dixie9000 it's likely you have the same issue. If that's the case, then his fix will work for you as well.

[shinmog]

I've added some documentation about "ansible_python_interpreter" to the docs for others, this will be on the main documentation page to help others if they have the same issue.

Furthermore, panos_op now fails gracefully if xmltodict is not present.

This will be present in the next collection release.

[rohitss]

I might be wrong here to comment but the mentioned error I am able to resolved on both of the below versions. This may be quick workaround

#ansible --version ansible [core 2.14.1] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python3.11/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/local/bin/ansible python version = 3.11.1 [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] (/usr/local/bin/python3.11) jinja version = 3.1.2 libyaml = True

#ansible --version ansible [core 2.13.7] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python3.8/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/local/bin/ansible python version = 3.8.12 [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] jinja version = 3.1.2 libyaml = True

#Solution:
1- You need to install pan-python & pan-os-python using pip for the current version of python which you are seeing ansible version output of yours then finally you have to run the below command & error should be gone.

pip3.6 install pan-python pip3.6 install pan-os-python