st2
st2 copied to clipboard
StackStorm
Upgrade python 3.6 -> 3.8 on CentOS/RedHat 8
SUMMARY
I installed st2-3.5.0-1.x86_64 on RedHat 8, the python version included in st2 is 3.6.8.
st2 3.5.0, on Python 3.6.8
But Python 3.6 is going to EOL in December 2021 https://www.python.org/dev/peps/pep-0494/#lifespan, is there plan to upgrade the dependent python to 3.7+ ?
STACKSTORM VERSION
st2 3.5.0, on Python 3.6.8
OS, environment, install method
OS: RedHat 8.4 Installed: online install
Steps to reproduce the problem
Show how to reproduce the problem, using a minimal test-case. Make sure to include any content (pack content - workflows, actions, etc.) which are needed to reproduce the problem.
Expected Results
Python version should be 3.7+
Actual Results
Python version is 3.6 which is going to EOL in December 2021
We build and ship a single .rpm package that works for both CentOS/RHEL 8. Are there any native python 3.7 or 3.8 packages available for installation in CentOS 8 and Red Hat 8?
@armab which .rpm package you mean? is it for stackstorm 3.5? And there should be no other python installed on this box.
When the StackStorm rpm package is installed on the box, it relies on python3-devel package as a dependency.
In other words, we install dependency that exists in the OS system. For CentOS8 it's python 3.6 at this moment.
The upstream OS should provide a python 3.7 or 3.8 for us to use it. Are there py3.7 or py3.8 native packages available for installation under CentOS/RHEL 8?
My OS is RedHat 8.4, and below are the existing python packages on the system before st2 install, there are python38-devel.x86_64 and python39-devel.x86_64 available.
[root@kishke1 ~]# yum search python|grep devel
platform-python-devel.i686 : Libraries and header files needed for Python development
platform-python-devel.x86_64 : Libraries and header files needed for Python development
python2-devel.x86_64 : Libraries and header files needed for Python 2 development
python2-tools.x86_64 : A collection of development tools included with Python 2
python3-idle.i686 : A basic graphical development environment for Python
python3-idle.x86_64 : A basic graphical development environment for Python
python3-werkzeug.noarch : The Swiss Army knife of Python web development
python36-devel.x86_64 : Libraries and header files needed for Python development
python38-devel.x86_64 : Libraries and header files needed for Python development
python38-idle.x86_64 : A basic graphical development environment for Python
python39-devel.x86_64 : Libraries and header files needed for Python development
python39-idle.x86_64 : A basic graphical development environment for Python
python2-cairo-devel.x86_64 : Libraries and headers for py2cairo
Then installed StackStorm, the st2 version showed python 3.6.8 with /opt/stackstorm/st2/lib/python3.6
[root@kishke1 lib]# st2 --version
st2 3.5.0, on Python 3.6.8
Thanks for more info @yypptest!
python38-devel.x86_64 is looking like a suitable package and it's available in CentOS 8 as well.
[vagrant@centos8 ~]$ repoquery -i python38-devel
CentOS Linux 8 - AppStream 2.9 MB/s | 9.5 MB 00:03
CentOS Linux 8 - BaseOS 1.8 MB/s | 7.5 MB 00:04
CentOS Linux 8 - Extras 9.0 kB/s | 10 kB 00:01
Name : python38-devel
Version : 3.8.6
Release : 3.module_el8.4.0+665+abc3a503
Architecture : x86_64
Size : 275 k
Source : python38-3.8.6-3.module_el8.4.0+665+abc3a503.src.rpm
Repository : appstream
Summary : Libraries and header files needed for Python development
URL : https://www.python.org/
License : Python
Description : This package contains the header files and configuration needed to compile
: Python extension modules (typically written in C or C++), to embed Python
: into other programs, and to make binary distributions for Python libraries.
:
: It also contains the necessary macros to build RPM packages with Python modules
: and 2to3 tool, an automatic source converter from Python 2.X.
:
: If you want to build an RPM against the python38 module, you also need to
: install the python38-rpm-macros package.
Nothing is blocking us on this. Sounds like a good upgrade candidate for the next release @StackStorm/tsc?
@armab Glad to see the python is to be upgraded in st2 3.7.0, thanks.
Before the st23.7.0 is released, could you suggest how should I upgrade the python in my local environment?
@yypptest You need to modify the rpm packaging process to rely on the new python38-devel python package as a requirement and build your own st2 packages.
The starting point is: https://github.com/StackStorm/st2-packages/blob/master/packages/st2/rpm/st2.spec#L17 .
How about opening a Pull Request there?
@armab I'm afraid we could not wait for the st2 3.7.0, we need to upgrade the python in st2 ASAP.
How should we upgrade python in an existing installed st2 env or is there parameter available during install to specify what version of python to be installed?
FYI we're building RHEL8-compatible packages in a CentOS8 environment. Based on @StackStorm/tsc maintainers meeting today https://github.com/StackStorm/community/issues/96#issuecomment-1005180964 and https://github.com/StackStorm/community/issues/88 plan as CentOS8 has reached its EOL in December 2021, we'll be dropping the support for original/old EL8 packages since st2 v3.7.0.
Instead, we'll start building the generic .rpm package based on RockyLinux (https://rockylinux.org/), which is a compatible community replacement for CentOS that (in theory) should work for RHEL8, Oracle Linux, Rocky Linux, and Alma Linux. I think that could be the point when we'll be able to switch the python 3.6 -> 3.8.
@yypptest If you're using ST2 in prod and have some capacity to help with that plan, have ideas, or perhaps could help with testing or feedback, - contributions or any involvement would be very welcome.
@stevemuskiewicz Thanks! The main discussion is happening in the https://github.com/StackStorm/community/issues/88
@armab - are we covering the python 3.6->3.8 upgrade in the CentOS8 -> Rocky etc discussion? I think the upgrade to python3.8 was going to be handled separately?
¯_(ツ)_/¯ Indeed, different topics, but tightly coupled.
@amanda11 I guess, it makes little sense to release Rocky st2.rpm with the python3.6 as a dependency which came EOL at the same time as CentOS8 (Dec 2021).
Otherwise, we could just use the old EOL'ed CentOS8 for generating the .rpm packages.
@armab May still be worth first testing Rocky with python 3.6, as if we have issues we wont know if it was move to python 3.8 on EL8 in general, or to do with moving towards Rocky. I agree both need to be in 3.7.0 release, but the python 3.8 move will require a lot more changes than moving to Rocky from CentOS8...