Python 3.8 support for Netbox 3.x on RHEL 7

[mtinberg]

I have reworked the playbook to use Python 3.8 from SCL on RHEL7 but haven't tested on other platforms such as Debian/Ubuntu or RHEL8.

There are some workarounds for not using a system-wide install of Python with Ansible and uWSGI, Ansible builtin template module imports the selinux library which does not seem to have a prebuilt package for SCL Python3.8 on RHEL7 the way that libselinux-python and libselinux-python3 support python27 and python36 on RHEL7, so I reset the ansible_python_interpreter around sections that use ansible.builtin.template when the OS is RHEL7 (or similar like CentOS or OracleLinux). I also allowed it to install the selinux library in the virtualenv but to use that as the ansible_python_interpreter would require building the venv earlier and further restructuring.

On that vein I did explicitly build the venv in its own command rather than having it implicitly happen as part of the first pip install as the implicit venv doesn't seem to upgrade and existing python36 venv to python38, it uses the version of python that the venv was originally built with. It may be more clear to just remove the existing venv before the first pip command, if you try to upgrade python using the same version of netbox, unlike how it'll build a new venv when building a new version of netbox.

The other workaround was to install uWSGI in the Python3.8 virtualenv and reference that in the systemd service file instead of trying to install it systemwide in /usr/local/bin/uwsgi as the systemwide one doesn't upgrade from python36 to python38 or have the right path information built into it when using Python3.8 from SCL on RHEL7. So on my system at least I end up with three versions of uwsgi, one installed from RPM in /usr/sbin/uwsgi, one installed from the playbook in python36 in /usr/local/bin/uwsgi and one installed from the playbook with python38 in /opt/netbox/current/venv-py3/bin/uwsgi

Take a look at the diff and tell me what you think, is this something you want?

[lae]

If I don't get a chance to look at this by the end of the month, feel free to ping me a reminder.

Of course, if anyone else is available, please review this (and possibly test on other distros) and give me your thoughts.

[br-olf]

@lae I'll take the duty to ping you regarding this issue. I currently fight with RHEL8 and would love to see a merge.

I'll checkout the merge request for myself and give you an update soonish but I'll be on vacation for 2 weeks so updates from myself will have to wait until may.

[mtinberg]

No worries, this has continued to work for me although I did make one change to run the mkdocs command before collecting the static files as it doesn't work the other way around 😉

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: Olaf Pichler @.> Sent: Wednesday, April 13, 2022 1:38 PM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Author @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

@laehttps://github.com/lae I'll take the duty to ping you regarding this issue. I currently fight with RHEL8 and would love to see a merge.

I'll checkout the merge request for myself and give you an update soonish but I'll be on vacation for 2 weeks so updates from myself will have to wait until may.

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1098365602, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UM2F6YEFQXR4KQ3JOPTVE4ID7ANCNFSM5IM2JLPQ. You are receiving this because you authored the thread.Message ID: @.***>

[ironiq]

@mtinberg just a note: i cannot see anywhere installing the "centos-release-scl" package on CentOS 7 to be able to install rh-python38 packages. I think the "centos-sclo-*" repos are not enabled by default everywhere. Or am i missing something?

[mtinberg]

That may well be an oversight where I already had the repo definition installed and didn't notice the need, IIUC the centos-release-scl package which contains the YUM repository definitions is in the extras repo, which is defined in the Centos-Base.repo file but may need to be enabled to install packages out of it.

Thanks for checking it over so we can work out the "worksforme(tm)" issues

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: ironiq @.> Sent: Monday, May 2, 2022 4:52 AM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Mention @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

@mtinberghttps://github.com/mtinberg just a note: i cannot see anywhere installing the "centos-release-scl" package on CentOS 7 to be able to install rh-python38 packages. I think the "centos-sclo-*" repos are not enabled by default everywhere. Or am i missing something?

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1114678219, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UM6B2CLNGFQVTQK7PIDVH6QXVANCNFSM5IM2JLPQ. You are receiving this because you were mentioned.Message ID: @.***>

[mtinberg]

I added a task to install_packages_yum.yml which I'll commit as soon as my test in the lab completes

• name: Install SCL repository (CentOS) yum: name: centos-release-scl state: installed enablerepo: extras when: "ansible_distribution == 'CentOS' and ansible_distribution_major_version == '7'"

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: ironiq @.> Sent: Monday, May 2, 2022 4:52 AM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Mention @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

@mtinberghttps://github.com/mtinberg just a note: i cannot see anywhere installing the "centos-release-scl" package on CentOS 7 to be able to install rh-python38 packages. I think the "centos-sclo-*" repos are not enabled by default everywhere. Or am i missing something?

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1114678219, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UM6B2CLNGFQVTQK7PIDVH6QXVANCNFSM5IM2JLPQ. You are receiving this because you were mentioned.Message ID: @.***>

[tyler-8]

IMO - any tasks related to installation of a Python binary need to be optional. Not every CentOS user wants to or can install the EPEL or centos-release-scl. Adding something like the existing var netbox_install_epel, such as netbox_install_centos_scl: False.

[ironiq]

@tyler-8 how will you provide python3.8 other way on redhat7-based distros? IMO building the package locally is even worse.

[tyler-8]

Ultimately, it doesn't matter if the binary comes from EPEL, SCL, a private repo, a local build-from-source, or any other means, it should be outside the scope of this role. As long as I provide a valid Python binary that meets the compatibility requirements of NetBox, this role should function.

All I'm suggesting in this instance is to make repository installs optional.

[mtinberg]

I get it, wrap the SCL install (like EPEL) in some optional flags, so the admin can install their own python37+ and that choice won't be overridden by installing epel/scl/etc. So we'll need to make sure we don't presume the path of the installed python as well, if the admin is providing it outside of this playbook

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: Tyler Bigler @.> Sent: Monday, May 2, 2022 11:33 AM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Mention @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

Ultimately, it doesn't matter if the binary comes from EPEL, SCL, a private repo, a local build-from-source, or any other means, it should be outside the scope of this role. As long as I provide a valid Python binary that meets the compatibility requirements of NetBox, this role should function.

All I'm suggesting in this instance is to make repository installs optional.

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1115100503, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UM45IJEPNKVERFTCJN3VH77VJANCNFSM5IM2JLPQ. You are receiving this because you were mentioned.Message ID: @.***>

[tyler-8]

So we'll need to make sure we don't presume the path of the installed python as well, if the admin is providing it outside of this playbook

I believe you should be fine as long as you're using netbox_python_binary and not hard-coding any paths (outside of defaults)

[mtinberg]

I've configured this path in redhat-7.yml as _netbox_python_binary: /opt/rh/rh-python38/root/usr/bin/python3.8 and I'd have to re-read the whole thing to make sure that isn't a bad way to do it as I don't recall offhand how data flows through this particular playbook and if there is a better place to configure this (make it conditional on whether a netbox_enable_scl: true flag is thrown elsewhere in the config like for epel).

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: Tyler Bigler @.> Sent: Monday, May 2, 2022 1:19 PM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Mention @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

So we'll need to make sure we don't presume the path of the installed python as well, if the admin is providing it outside of this playbook

I believe you should be fine as long as you're using netbox_python_binary and not hard-coding any paths (outside of defaults)

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1115213267, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UMY6PWUCXO6X54NHXA3VIAMEPANCNFSM5IM2JLPQ. You are receiving this because you were mentioned.Message ID: @.***>

[mtinberg]

There is bit of weirdness if I recall right because when ansible logs into the system it may be using the /usr/bin/python or /usr/bin/python3 interpreter but neither of those can manipulate the venv for python37+ using native modules without changing which interpreter ansible itself is using on the managed system, so there is a lot of switching back and forth the ansible_python variable, esp when the system-wide install comes with the selinux python module that ansible expects to use when manipulating files, but the scl rh-python38 doesn't come with that for some reason.

anyway I'd need to make sure that an admin-supplied python is used in the same way and it doesn't accidently switch to an scl installed python that doesn't exist when it's not enabled.

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: Mark Tinberg @.> Sent: Monday, May 2, 2022 2:07 PM To: lae/ansible-role-netbox @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

I've configured this path in redhat-7.yml as _netbox_python_binary: /opt/rh/rh-python38/root/usr/bin/python3.8 and I'd have to re-read the whole thing to make sure that isn't a bad way to do it as I don't recall offhand how data flows through this particular playbook and if there is a better place to configure this (make it conditional on whether a netbox_enable_scl: true flag is thrown elsewhere in the config like for epel).

—

Mark Tinberg @.***>

Division of Information Technology-Network Services

University of Wisconsin-Madison

---

From: Tyler Bigler @.> Sent: Monday, May 2, 2022 1:19 PM To: lae/ansible-role-netbox @.> Cc: Mark Tinberg @.>; Mention @.> Subject: Re: [lae/ansible-role-netbox] Python 3.8 support for Netbox 3.x on RHEL 7 (PR #138)

So we'll need to make sure we don't presume the path of the installed python as well, if the admin is providing it outside of this playbook

I believe you should be fine as long as you're using netbox_python_binary and not hard-coding any paths (outside of defaults)

— Reply to this email directly, view it on GitHubhttps://github.com/lae/ansible-role-netbox/pull/138#issuecomment-1115213267, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAS7UMY6PWUCXO6X54NHXA3VIAMEPANCNFSM5IM2JLPQ. You are receiving this because you were mentioned.Message ID: @.***>