salt icon indicating copy to clipboard operation
salt copied to clipboard

Published 20 hours ago • verified publisher icon saltstack

[BUG] salt-minion service fails to start on fresh 3005-1 installation with error: pkg_resources.DistributionNotFound: The 'salt==3005' distribution was not found and is required by the application

Open stonith79 opened this issue 3 years ago • 4 comments

Description

Salt-minion service fails to start after fresh installation via yum on RHEL 8.6, with error from below:

pkg_resources.DistributionNotFound: The 'salt==3005' distribution was not found and is required by the application

salt-minion.service - The Salt Minion Loaded: loaded (/usr/lib/systemd/system/salt-minion.service; disabled; vendor preset: disabled)

Active: failed (Result: exit-code) since Tue 2022-09-20 13:31:19 IST; Process: 46419 ExecStart=/usr/bin/salt-minion (code=exited, status=1/FAILURE) Main PID: 46419 (code=exited, status=1/FAILURE) Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 583, in _build_master Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: ws.require(requires) Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 900, in require Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: needed = self.resolve(parse_requirements(requirements)) Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 786, in resolve Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: raise DistributionNotFound(req, requirers) Sep 20 13:31:19 hostname.mydomain salt-minion[46419]: pkg_resources.DistributionNotFound: The 'salt==3005' distribution was not found and is required by the application Sep 20 13:31:19 hostname.mydomain systemd[1]: salt-minion.service: Main process exited, code=exited, status=1/FAILURE Sep 20 13:31:19 hostname.mydomain systemd[1]: salt-minion.service: Failed with result 'exit-code'. Sep 20 13:31:19 hostname.mydomain systemd[1]: Failed to start The Salt Minion.

Setup

VMs running on VMware Salt-minion: RHEL8.6 / 4.18.0-372.9.1.el8.x86_64 / 3005-1 / python 3.8 Salt-master: RHEL 7.9 / 3.10.0-1160.76.1.el7.x86_64 /3005-1 / python 3.6

Master conf:

default_include: master.d/*.conf interface: IP ipv6: False publish_port: 4505 user: root enable_ssh_minions: True ret_port: 4506 pidfile: /var/run/salt-master.pid root_dir: /srv/salt/ pki_dir: /etc/salt/pki/master cachedir: /var/cache/salt/master/jobs keep_jobs: 120 gather_job_timeout: 30 timeout: 30 auto_accept: True top_file_merging_strategy: same gitfs_provider: pygit2 log_file: /var/log/salt/master log_level: debug log_level_logfile: debug

Steps to Reproduce the behavior

Install salt-minion via yum latest.repo Start salt-minion service

Expected behavior Salt-minion service starts without errors

Versions Report

salt --versions-report

Salt Version: Salt: 3005

Dependency Versions: cffi: 1.15.0 cherrypy: Not Installed dateutil: Not Installed docker-py: Not Installed gitdb: 4.0.9 gitpython: 3.1.20 Jinja2: 2.11.1 libgit2: 0.28.3 M2Crypto: 0.35.2 Mako: Not Installed msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.12.0 pygit2: 1.0.0 Python: 3.6.8 (default, Aug 13 2020, 07:46:32) python-gnupg: Not Installed PyYAML: 3.13 PyZMQ: 18.0.1 smmap: 5.0.0 timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.1.4

Salt Extensions: SSEAPE: 8.5.0.6

System Versions: dist: rhel 7.9 Maipo locale: ANSI_X3.4-1968 machine: x86_64 release: 3.10.0-1160.76.1.el7.x86_64 system: Linux version: Red Hat Enterprise Linux Server 7.9 Maipo

stonith79 avatar Sep 20 '22 13:09 stonith79

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

welcome[bot] avatar Sep 20 '22 13:09 welcome[bot]

Possible duplicate of #62681

Did you try using the bootstrap script at any point?

OrangeDog avatar Sep 20 '22 14:09 OrangeDog

Nope, we’ve always been problem free using yum

stonith79 avatar Sep 20 '22 15:09 stonith79

@stonith79 This indeed looks to be a duplicate of #62681. I see from above that you state Python 3.8 on the salt-minion. For classic (regular) packages to function on RHEL platforms, it is required that the default OS provided Python be provided, in this case Python 3.6.8 is the default OS version of Python on RHEL 8 / Centos 8 Stream. Also see https://github.com/saltstack/salt/issues/62594 and https://github.com/saltstack/salt/issues/61346.

Note: Alma 8 works fine with using 'alternatives --config python3' and selecting the version of Python 3 desired.

Please confirm that you are using Python 3.8 instead of default Python 3.6 on the RHEL 8 system.

Already talked with documentation to add statement stating 'need default OS python version for classic packaging on Redhat systems'. Didn't need it in the past, but with Py 3.6 being EOL now running into users selecting later version of Python. This is why onedir was developed, so that Salt could operate on OS's whose versions of Python were EOL, for example: RHEL 7 & 8., and even RHEL 9 with Py 3.9 is no longer getting updates (then they do 3.9.14 :) )

Doc request for stating above https://gitlab.com/saltstack/open/docs/salt-install-guide/-/issues/52

dmurphy18 avatar Sep 20 '22 17:09 dmurphy18

@dmurphy18, sorry for the delay.

I'm using 3.8.12, see below, new VM on VMware, rhel 8.6. Our salt-minion package comes from: https://repo.saltproject.io/py3/redhat/8/x86_64/latest

[root@hostname ~]# uptime 11:10:52 up 1:03, 1 user, load average: 0.02, 0.03, 0.01 [root@hostname ~]# which python3 /usr/bin/python3 [root@hostname ~]# python3 -V Python 3.8.12 [root@hostname ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.6 (Ootpa) [root@hostname ~]# which salt-minion /usr/bin/salt-minion [root@hostname ~]# grep python /usr/bin/salt-minion #!/usr/bin/python3 [root@hostname ~]# systemctl status salt-minion.service ● salt-minion.service - The Salt Minion Loaded: loaded (/usr/lib/systemd/system/salt-minion.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2022-09-22 10:11:51 IST; 1h 0min ago Docs: man:salt-minion(1) file:///usr/share/doc/salt/html/contents.html https://docs.saltproject.io/en/latest/contents.html Process: 23904 ExecStart=/usr/bin/salt-minion (code=exited, status=1/FAILURE) Main PID: 23904 (code=exited, status=1/FAILURE)

Sep 22 10:11:51 hostanme salt-minion[23904]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 583, in _build_master Sep 22 10:11:51 hostanme salt-minion[23904]: ws.require(requires) Sep 22 10:11:51 hostanme salt-minion[23904]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 900, in require Sep 22 10:11:51 hostanme salt-minion[23904]: needed = self.resolve(parse_requirements(requirements)) Sep 22 10:11:51 hostanme salt-minion[23904]: File "/usr/lib/python3.8/site-packages/pkg_resources/init.py", line 786, in resolve Sep 22 10:11:51 hostanme salt-minion[23904]: raise DistributionNotFound(req, requirers) Sep 22 10:11:51 hostanme salt-minion[23904]: pkg_resources.DistributionNotFound: The 'salt==3005' distribution was not found and is required by> Sep 22 10:11:51 hostanme systemd[1]: salt-minion.service: Main process exited, code=exited, status=1/FAILURE Sep 22 10:11:51 hostanme systemd[1]: salt-minion.service: Failed with result 'exit-code'. Sep 22 10:11:51 hostanme systemd[1]: Failed to start The Salt Minion.

[root@hostanme ~]# alternatives --config python3

There is 1 program that provides 'python3'.

Selection Command ----------------------------------------------- *+ 1 /usr/bin/python3.8

Enter to keep the current selection[+], or type selection number: [root@hostanme ~]# alternatives --display python3 python3 - status is auto. link currently points to /usr/bin/python3.8 /usr/bin/python3.8 - priority 3800 slave easy_install-3: /usr/bin/easy_install-3.8 slave pip-3: /usr/bin/pip-3.8 slave pip3: /usr/bin/pip3.8 slave pydoc-3: /usr/bin/pydoc3.8 slave pydoc3: /usr/bin/pydoc3.8 slave python3-man: /usr/share/man/man1/python3.8.1.gz Current `best' version is /usr/bin/python3.8. [root@hostanme ~]#

stonith79 avatar Sep 22 '22 10:09 stonith79

@stonith79 As stated in https://github.com/saltstack/salt/issues/62716#issuecomment-1252679550, with classic packaging you need to use the default OS supplied version of Python, which is 3.6.8.

In order to use Salt with Python 3.8 on Redhat 8 family, please use the onedir version of Salt 3005, available from https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-by-operating-system/centos.html#install-onedir-packages-of-salt-on-centos-8

Closing this, since normal operation.

dmurphy18 avatar Sep 22 '22 14:09 dmurphy18

So just to add to the document I am also just experiencing this issue on our RHEL8 instance we just spun up. @OrangeDog I specifically did use the bootstrap. Here is what i ran

wget -O bootstrap-salt.sh https://bootstrap.saltstack.com
sudo sh bootstrap-salt.sh -X -x python3

Using the bootstrap I am getting the following error from salt

Oct 05 10:03:36 x systemd[1]: Starting The Salt Minion...
Oct 05 10:03:37 x salt-minion[29854]: Traceback (most recent call last):
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/bin/salt-minion", line 6, in <module>
Oct 05 10:03:37 x salt-minion[29854]:     from pkg_resources import load_entry_point
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     def _initialize_master_working_set():
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     f(*args, **kwargs)
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     working_set = WorkingSet._build_master()
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     ws.require(__requires__)
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     needed = self.resolve(parse_requirements(requirements))
Oct 05 10:03:37 x salt-minion[29854]:   File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", lin>
Oct 05 10:03:37 x salt-minion[29854]:     raise DistributionNotFound(req, requirers)
Oct 05 10:03:37 x salt-minion[29854]: pkg_resources.DistributionNotFound: The 'salt==3005.1' distribution was >
Oct 05 10:03:38 x systemd[1]: salt-minion.service: Main process exited, code=exited, status=1/FAILURE
Oct 05 10:03:38 x systemd[1]: salt-minion.service: Failed with result 'exit-code'.
Oct 05 10:03:38 x systemd[1]: Failed to start The Salt Minion.

@dmurphy18 just to follow up on your comment ( https://github.com/saltstack/salt/issues/62716#issuecomment-1252679550 ) since I am having the same error I would include my response. Our build OS build is templated and all systems get the python38 module installed using the following command

dnf -q -y module install python38:3.8/common

and to confirm the install

yum info python38
Last metadata expiration check: 0:11:20 ago on Wed 05 Oct 2022 10:51:54 AM EDT.
Installed Packages
Name         : python38
Version      : 3.8.12
Release      : 1.module+el8.6.0+12642+c3710b74
Architecture : x86_64
Size         : 24 k
Source       : python38-3.8.12-1.module+el8.6.0+12642+c3710b74.src.rpm
Repository   : @System
From repo    : rhel-8-appstream-rhui-rpms
Summary      : Interpreter of the Python programming language
URL          : https://www.python.org/
License      : Python
Description  : Python is an accessible, high-level, dynamically typed, interpreted programming
             : language, designed with an emphasis on code readability.
             : It includes an extensive standard library, and has a vast ecosystem of
             : third-party libraries.
             : 
             : The python38 package provides the "python3" executable: the reference
             : interpreter for the Python language, version 3.
             : The majority of its standard library is provided in the python38-libs package,
             : which should be installed automatically along with python38.
             : The remaining parts of the Python standard library are broken out into the
             : python38-tkinter and python38-test packages, which may need to be installed
             : separately.
             : 
             : Documentation for Python is provided in the python38-docs package.
             : 
             : Packages containing additional libraries for Python are generally named with
             : the "python38-" prefix.
             : 
             : For the unversioned "python" executable, see manual page "unversioned-python".

I will test out the onedir version as presented by @dmurphy18 and see if that works.

ssoto2 avatar Oct 05 '22 15:10 ssoto2

@dmurphy18 looking at the onedir instructions install steps. It

sudo rpm --import https://repo.saltproject.io/salt/py3/redhat/8/x86_64/3005/SALTSTACK-GPG-KEY.pub
curl -fsSL https://repo.saltproject.io/salt/py3/redhat/8/x86_64/3005.repo | sudo tee /etc/yum.repos.d/salt.repo

Is locking the repo at 3005 is there any plan to have a 'latest' repo for the onedir so we can still receive updates (3006..)?

Also, will there be a bootstap that will work for the onedir soon?

ssoto2 avatar Oct 05 '22 15:10 ssoto2

@ssoto2 Here is the documentation that you are looking for on 'latest' for onedir RHEL 8, and yes, there is a bootstrap for it, please look at the latest documentation for bootstrap, it should contain what you need, esp. since a number of issue have been fixed, esp. with the release of Salt 3005.1

Lastly adding to a closed issue with questions which are not on topic of that closed issue, is not the best way to get an issue addressed. Please in future open a new issue, esp. since you are inquiring about install using 'latest', and the question could easily have been asked in the Community Slack. https://saltstackcommunity.slack.com/archives/C8E7RJCPN

dmurphy18 avatar Oct 05 '22 15:10 dmurphy18