saltstack
[BUG] Augeas module fails to load on EL9 3005~rc2-1
Description The augeas module/state does not appear to be getting loaded on EL9 3005~rc2-1.
Setup Installed the RC1~2 RPMs on a baremetal workstation running Rocky 9.0:
> rpm -qa | grep salt
salt-3005~rc2-1.el9.x86_64
salt-minion-3005~rc2-1.el9.x86_64
Steps to Reproduce the behavior The simplest way I've found to verify that the module is loaded is to run the following command and verify that the output is not empty. (This can be run without connecting to the server):
> salt-call --local sys.doc | grep augeas
Expected behavior If the augeas module is loaded correctly, I'd expect to see the augeas module documentation referenced. Instead, nothing is returned.
For comparison, on an EL8/3004.2 insallation, the following is returned:
> salt-call --local sys.doc | grep augeas
augeas.execute:
salt '*' augeas.execute /files/etc/redis/redis.conf \
augeas.get:
Get a value for a specific augeas path
salt '*' augeas.get /files/etc/hosts/1/ ipaddr
augeas.ls:
salt '*' augeas.ls /files/etc/passwd
augeas.match:
salt '*' augeas.match /files/etc/services/service-name ssh
augeas.remove:
salt '*' augeas.remove \
augeas.setvalue:
Set a value for a specific augeas path
salt '*' augeas.setvalue /files/etc/hosts/1/canonical localhost
salt '*' augeas.setvalue /files/etc/hosts/01/ipaddr 192.168.1.1 \
salt '*' augeas.setvalue prefix=/files/etc/sudoers/ \
augeas.tree:
salt '*' augeas.tree /files/etc/
Versions Report
salt-call --local --versions-report
Salt Version:
Salt: 3005rc2
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.0
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.17
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
Python: 3.9.13 (main, Apr 10 2020, 00:00:00)
python-gnupg: 0.4.8
PyYAML: 5.4.1
PyZMQ: 23.2.0
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: rocky 9.0 Blue Onyx
locale: utf-8
machine: x86_64
release: 5.14.0-70.17.1.el9_0.x86_64
system: Linux
version: Rocky Linux 9.0 Blue Onyx
This module requires the
augeasPython module.
How have you installed it? Note that the new onedir packages do not use the system Python.
Admittedly, I'm unsure what that really requires with the new packaging.
From the system Python, augeas seems to be functional.
> rpm -q python3-augeas
python3-augeas-0.5.0-25.el9.noarch
> python --version
Python 3.9.10
> python -c "from augeas import Augeas"
(succeeds)
@imjustvisiting With the 3005 release we're moving towards having onedir packages, where everything that Salt uses will be contained in a single directory, and will use a provided version of Python so we won't be using the system provided Python. Because of this, any Python packages provided by your operating system will not be accessible by Salt. To use the augeas module you'll need to install the augeas library into the Salt onedir environment using the salt-pip install python-augeas command.
It would be salt-pip install augeas.
python-augeas is the rpm package name.
Ah. Okay. Hrmm...
So for offline/air-gapped systems, I need to mirror PyPI now? What's the line between "native" modules and third-party? Is there a good way to know which modules/states no longer work without manually installing modules?
A pip install seemed to work okay once I reconnected to the Internet. (Though pip is slightly grumpy about version resolution.)
Also, I think python-augeas is the name of the pip package as well... (not 'augeas')
> salt-pip install augeas
ERROR: Could not find a version that satisfies the requirement augeas (from versions: none)
ERROR: No matching distribution found for augeas
Distribution augeas was not found installed
> salt-pip install python-augeas
Collecting python-augeas
Using cached python_augeas-1.1.0-py3-none-any.whl
Collecting cffi>=1.0.0
Using cached cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pycparser, cffi, python-augeas
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
salt 3005rc2 requires cffi==1.14.6, but you have cffi 1.15.1 which is incompatible.
salt 3005rc2 requires pycparser==2.17, but you have pycparser 2.21 which is incompatible.
Successfully installed cffi-1.15.1 pycparser-2.21 python-augeas-1.1.0
> salt-call --local sys.doc | grep augeas
augeas.execute:
salt '*' augeas.execute /files/etc/redis/redis.conf \
augeas.get:
Get a value for a specific augeas path
salt '*' augeas.get /files/etc/hosts/1/ ipaddr
augeas.ls:
salt '*' augeas.ls /files/etc/passwd
augeas.match:
salt '*' augeas.match /files/etc/services/service-name ssh
augeas.remove:
salt '*' augeas.remove \
augeas.setvalue:
Set a value for a specific augeas path
salt '*' augeas.setvalue /files/etc/hosts/1/canonical localhost
salt '*' augeas.setvalue /files/etc/hosts/01/ipaddr 192.168.1.1 \
salt '*' augeas.setvalue prefix=/files/etc/sudoers/ \
augeas.tree:
salt '*' augeas.tree /files/etc/
python-augeas is the name of the pip package as well...
I stand corrected. So much for consistency, eh.
I need to mirror PyPI now?
salt-pip should be able to install from many different sources, same as pip. There should only be a small set of packages you need, which you could distribute manually as .whl files, e.g. using the Salt fileserver.
So just to be clear:
If I operate an enterprise in an offline environment it is now on me as the user to go and find all the python modules that are used by all the states and formulas that are in use in my environment, including third-party states and formulas that I may not have written, that aren't included in whatever saltproject has deemed as "core functionality"?
If this is the case, why not create an additional package that has these modules in it, especially if the set really is as small as you believe? That way we know that the modules that you provide are compatible with the python that you also provide in the onedir package, and we don't end up in dependency hell trying to match up modules with whatever python shipped with onedir salt.
I recognize that this is probably not your most important use case to consider, but not everyone wants to hang their systems off the internet or push a bunch of whl files around to the minions that need them.
You did have to do that anyway, but it just happened that you had python3-augeas installed already.
I agree it's not a very good upgrade experience. A migration tool was suggested (#62361), but abandoned.
Yeah, I guess I'm not too wild about having to manually manage and resolve packages either. For those of us with a lot of disconnected systems, this is kinda gross. Requiring two different package managers in order to install a single application (dnf and pip) seems to fly in the face of "simplified" installation.
Seems like onedir is fairly opinionated about the Python and packages that it is bundling. Why not include the packages needed for all of the states/modules that are mentioned in the official documentation to continue the "batteries included" experience?
If the concern is the salt package being too heavy, maybe kicking out the "extra" modules into a separate/optional salt-modules package would be preferable to requiring the end user to manually curate a handful of whl files.
As far as this bug goes, I think the expected-behavior tag resolves it. Thanks again for your help!
@imjustvisiting with regard to
concern is the salt package being too heavy, maybe kicking out the "extra" modules into a separate/optional salt-modules package
There is an ongoing project to move modules to salt extensions https://github.com/saltstack/salt-extension and https://github.com/saltstack/salt-enhancement-proposals/pull/67
Just for some flavor (and one I would recommend in an airgapped system) - it is possible to follow the instructions to build your own onedir (nee tiamat) package of Salt, and define the dependencies that you want to include. It's not really that much more of a lift than any other approach, and has the added benefit of putting the entire packaging under your control -- if you verify the hashes, etc. it gives you a lot more of an audit trail.
I'll go ahead and close this issue out then -- thanks!
