saltstack
[BUG] Regression in 3006.11 vs. 3006.10 - 'augeas.change' was not found in SLS
Description
When running a state augeas.change with salt-minion 3006.11 it fails with this message:
ID: <redacted>
Function: augeas.change
Result: False
Comment: State 'augeas.change' was not found in SLS '<redacted>'
Reason: 'augeas.change' is not available.
The exact same configuration in working fine with 3006.10 (salt master is still in 3006.11 in this case) I noticed also that the latest version of 3007 is also having the issue (the first versions of 3007 are working fine).
Setup
- [x] on-prem machine
- [ ] VM (Virtualbox, KVM, etc. please specify)
- [ ] VM running on a cloud service, please be explicit and add details
- [ ] container (Kubernetes, Docker, containerd, etc. please specify)
- [ ] or a combination, please be explicit
- [ ] jails if it is FreeBSD
- [ ] classic packaging
- [x] onedir packaging
- [ ] used bootstrap to install
Steps to Reproduce the behavior
Be sure to install the python augeas module with this command line:
salt-pip install python-augeas (with sudo in front of it in my case)
Example of state to reproduce the issue (any augeas.change would reproduce the issue):
<redacted>:
augeas.change:
- lens: Hosts.lns
- context: /files/etc/hosts
- changes:
- set 100/ipaddr '192.168.0.44'
- set 100/canonical 'printer'
- unless: grep '192.168.0.44' /etc/hosts
Expected behavior
augeas.change should be found and outpout something like: Name: <redacted> - Function: augeas.change - Result: Clean - Started: 12:20:54.014861 - Duration: 16.048 ms
Screenshots N/A
Versions Report
salt --versions-report
Version 3006.11 (failing)
Salt Version:
Salt: 3006.11
Python Version:
Python: 3.10.17 (main, May 11 2025, 04:07:13) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
cryptography: 42.0.5
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.6
libgit2: Not Installed
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.19.2
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: debian 12 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-37-amd64
system: Linux
version: Debian GNU/Linux 12 bookworm
Version 3006.10 (working fine)
Salt Version:
Salt: 3006.10
Python Version:
Python: 3.10.16 (main, Mar 6 2025, 02:23:15) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
cryptography: 42.0.5
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.6
libgit2: Not Installed
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.18.1
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: debian 12 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-37-amd64
system: Linux
version: Debian GNU/Linux 12 bookworm
Additional context Add any other context about the problem here.
The actual error is this:
# /opt/saltstack/salt/bin/python3 -c "from augeas import Augeas"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/saltstack/salt/extras-3.10/augeas/__init__.py", line 35, in <module>
from _augeas import ffi, lib
ImportError: /opt/saltstack/salt/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libicuuc.so.74)
Looks like with the 3006.11 packages, also relenv was updated from 0.18.1 to 0.19.2.
@dwoz Maybe this issue is about the changes around libstdc++ in relenv 0.19.0 ?:
https://github.com/saltstack/relenv/blob/8bbacbe90525bdbe66d0d9c5f1548d9efec3de60/CHANGELOG.md?plain=1#L29-L32
Result of /opt/saltstack/salt/bin/python3 -c "from augeas import Augeas"
on 3006.10
<no output produced>
on 3006.12
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/saltstack/salt/extras-3.10/augeas/__init__.py", line 35, in <module>
from augeas.ffi import ffi, lib
File "/opt/saltstack/salt/extras-3.10/augeas/ffi.py", line 47, in <module>
lib = ffi.dlopen("augeas")
File "/opt/saltstack/salt/lib/python3.10/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/opt/saltstack/salt/lib/python3.10/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/opt/saltstack/salt/lib/python3.10/site-packages/cffi/api.py", line 828, in _load_backend_lib
return backend.load_library(path, flags)
OSError: cannot load library 'libaugeas.so.0': /opt/saltstack/salt/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libicuuc.so.74)
How is
python-augeasbeing installed?
with the command salt-pip install python-augeas
@doc75
Does installing with this command RELENV_BUILDENV=1 salt-pip install python-augeas resolve the issue?
I first uninstalled it with salt-pip uninstall python-augeas
Then reinstall it with RELENV_BUILDENV=1 salt-pip install python-augeas
I have the exact same message reported by @hurzhurz :
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/saltstack/salt/extras-3.10/augeas/__init__.py", line 35, in <module>
from _augeas import ffi, lib
ImportError: /opt/saltstack/salt/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libicuuc.so.74)
@doc75 The reinstall worked (and had the same result) because you already head a built wheel in the cache, RELENV_BUILDENV had no effect.
@dwoz installing/building with RELENV_BUILDENV=1 doesn't work because python-augeas requires some system libs (libaugeas, libicu, ...?) that it doesn't find this way:
# RELENV_BUILDENV=1 salt-pip install --no-cache-dir python-augeas
Collecting python-augeas
Downloading python-augeas-1.2.0.tar.gz (99 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.4/99.4 kB 3.6 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: cffi>=1.0.0 in /opt/saltstack/salt/lib/python3.10/site-packages (from python-augeas) (1.14.6)
Requirement already satisfied: pycparser in /opt/saltstack/salt/lib/python3.10/site-packages (from cffi>=1.0.0->python-augeas) (2.21)
Building wheels for collected packages: python-augeas
Building wheel for python-augeas (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
/opt/saltstack/salt/lib/python3.10/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/augeas
copying augeas/ffi.py -> build/lib.linux-x86_64-cpython-310/augeas
copying augeas/__init__.py -> build/lib.linux-x86_64-cpython-310/augeas
running build_ext
generating cffi module 'build/temp.linux-x86_64-cpython-310/_augeas.c'
creating build/temp.linux-x86_64-cpython-310
building '_augeas' extension
creating build/temp.linux-x86_64-cpython-310/build
creating build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310
/root/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wno-coverage-mismatch -g -I/opt/saltstack/salt/include -I/opt/saltstack/salt/include/readline -I/opt/saltstack/salt/include/ncursesw -I/root/.local/relenv/toolchain/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/include -fPIC -I/usr/include/libxml2 -I/opt/saltstack/salt/include/python3.10 -I/opt/saltstack/salt/include -c build/temp.linux-x86_64-cpython-310/_augeas.c -o build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/_augeas.o
build/temp.linux-x86_64-cpython-310/_augeas.c:570:25: fatal error: augeas.h: No such file or directory
570 | #include <augeas.h>
| ^~~~~~~~~~
compilation terminated.
error: command '/root/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for python-augeas
Running setup.py clean for python-augeas
Failed to build python-augeas
ERROR: Could not build wheels for python-augeas, which is required to install pyproject.toml-based projects
[notice] A new release of pip is available: 23.3.2 -> 25.1.1
[notice] To update, run: /opt/saltstack/salt/bin/python3.10 -m pip install --upgrade pip
@hurzhurz it seems that you are seeing my answer before I edited it 😢
Indeed I realized that I was missing gcc and libaugeas-dev. I added them and the install went fine, but then the error is still there (the same as the one you reported earlier by just doing the import.
Anything else I could test ?
@doc75 well, I got your unedited comment via email notification, but I was not referring to it.
I see when you first tried it with RELENV_BUILDENV=1 salt-pip install python-augeas, you got this error, correct?:
error: command '/root/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc' failed: No such file or directory
That can be solved by: /opt/saltstack/salt/bin/relenv toolchain fetch
But the error from my last comment, where it doesn't find augeas.h, will probably remain.
Important for testing: after salt-pip uninstall python-augeas also do salt-pip cache purge befor installing again
Without RELENV_BUILDENV=1, you need gcc and libaugeas-dev in your system for installation.
And with it, they need to be in the relenv build environment. While gcc can be gathered by the mention fetch toolchain command, I don't know how to get libaugeas-dev in there.
@dwoz I noticed it works without RELENV_BUILDENV=1 under Ubuntu 25.04.
But under Ubuntu 24.04, importing the installed modul fails with the error about libstdc++.so.6.
Here is what I just did:
root@host ~ # /opt/saltstack/salt/bin/relenv toolchain fetch
Toolchain directory: /root/.local/relenv/toolchain
Downloading https://packages.broadcom.com/artifactory/saltproject-generic/relenv/0.17.4/toolchain/x86_64/x86_64-linux-gnu.tar.xz -> /root/.local/relenv/toolchain/x86_64-linux-gnu.tar.xz
root@host ~ # salt-pip uninstall python-augeas Found existing installation: python-augeas 1.2.0
Uninstalling python-augeas-1.2.0:
Would remove:
/opt/saltstack/salt/extras-3.10/_augeas.abi3.so
/opt/saltstack/salt/extras-3.10/augeas/*
/opt/saltstack/salt/extras-3.10/python_augeas-1.2.0.dist-info/*
Proceed (Y/n)? y
Successfully uninstalled python-augeas-1.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what y
ou are doing and want to suppress this warning.
root@host ~ # salt-pip cache purge
Files removed: 16 (2.1 MB)
# RELENV_BUILDENV=1 salt-pip install python-augeas
Collecting python-augeas
Downloading python-augeas-1.2.0.tar.gz (99 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: cffi>=1.0.0 in /opt/saltstack/salt/lib/python3.10/site-packages (from python-augeas) (1.14.6)
Requirement already satisfied: pycparser in /opt/saltstack/salt/lib/python3.10/site-packages (from cffi>=1.0.0->python-augeas) (2.21)
Building wheels for collected packages: python-augeas
DEPRECATION: Building 'python-augeas' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build
-isolation`), or adding a `pyproject.toml` file to the source tree of 'python-augeas'. Discussion can be found at https://github.com/pypa/pip/issues/6334
Building wheel for python-augeas (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
/opt/saltstack/salt/lib/python3.10/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/augeas
copying augeas/__init__.py -> build/lib.linux-x86_64-cpython-310/augeas
copying augeas/ffi.py -> build/lib.linux-x86_64-cpython-310/augeas
running build_ext
generating cffi module 'build/temp.linux-x86_64-cpython-310/_augeas.c'
creating build/temp.linux-x86_64-cpython-310
building '_augeas' extension
creating build/temp.linux-x86_64-cpython-310/build
creating build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310
/root/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wno-coverage-mismatch -g -I/opt/saltstack/salt/include -I/opt/saltstack/salt/include/readline -I/opt/saltstack/salt/include/ncursesw -I/root/.local/relenv/toolchain
/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/include -fPIC -I/opt/saltstack/salt/include/python3.10 -I/opt/saltstack/salt/include -c build/temp.linux-x86_64-cpython-310/_augeas.c -o build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/_augeas.o
build/temp.linux-x86_64-cpython-310/_augeas.c:570:25: fatal error: augeas.h: No such file or directory
570 | #include <augeas.h>
| ^~~~~~~~~~
compilation terminated.
error: command '/root/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for python-augeas
Running setup.py clean for python-augeas
Failed to build python-augeas
ERROR: Failed to build installable wheels for some pyproject.toml based projects (python-augeas)
I don't know how to get libaugeas-dev in there
I don't know either :-(
I tried with salt 3006.14 and Ubuntu 24.04 but I still get the libstdc++.so.6 error.
I have the same problem starting with Saltstack 3007.2 under Debian 12 (3007.1 is good).
Without libaugeas-dev salt-pip install python-augeas complains about the missing augeas.h.
With libaugeas-dev I get the GLIBCXX_3.4.30 error.
It seems that Debian 12 libicuuc.so.72 is not compatible with libstdc++.so.6 shipped by saltstack.
I have fixed this error by removing salt's libstdc++.so.6 so that the system's version is used:
sudo rm /opt/saltstack/salt/lib/libstdc++.so.6
This might have other side effects since I guess it has a reason that saltstack ships its own version, but at least on my systems this seems to solve the issue.
I had this issue as well on Debian 12 with 3006.14.
[ERROR ] State 'augeas.change' was not found in SLS 'baseserver'
Reason: 'augeas.change' is not available.
Resolved it with the information from this issue, thanks 🙏
sudo apt install build-essential
sudo apt install libaugeas-dev
sudo rm /opt/saltstack/salt/lib/libstdc++.so.6
sudo salt-pip uninstall python-augeas
sudo salt-pip cache purge
sudo salt-pip install python-augeas
sudo systemctl restart salt-minion.service
As @Cybso said in the previous post. No idea if the removal of libstdc++.so.6 has any side effects or issues. Everything seems to work okay.
