pyhss Python issues on Ubuntu 24.04 - PyHSS requirements cannot be installed

root@testhss:/etc/pyhss# pip3 install -r requirements.txt error: externally-managed-environment

× This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. root@testhss:/etc/pyhss# pip3 install -r requirements.txt --break-system-packages Collecting aiohttp==3.8.5 (from -r requirements.txt (line 1)) Using cached aiohttp-3.8.5.tar.gz (7.4 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting Flask==2.2.3 (from -r requirements.txt (line 2)) Using cached Flask-2.2.3-py3-none-any.whl.metadata (3.9 kB) Collecting flask_restx==1.1.0 (from -r requirements.txt (line 3)) Using cached flask_restx-1.1.0-py2.py3-none-any.whl.metadata (9.0 kB) Collecting Jinja2==3.1.2 (from -r requirements.txt (line 4)) Using cached Jinja2-3.1.2-py3-none-any.whl.metadata (3.5 kB) Collecting mongo==0.2.0 (from -r requirements.txt (line 5)) Using cached mongo-0.2.0.tar.gz (4.6 kB) Preparing metadata (setup.py) ... done Collecting prometheus_client==0.16.0 (from -r requirements.txt (line 6)) Using cached prometheus_client-0.16.0-py3-none-any.whl.metadata (22 kB) Collecting pycryptodome==3.17 (from -r requirements.txt (line 7)) Using cached pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB) Collecting pymongo==4.3.3 (from -r requirements.txt (line 8)) Using cached pymongo-4.3.3.tar.gz (814 kB) Preparing metadata (setup.py) ... done Collecting pysctp==0.7.2 (from -r requirements.txt (line 9)) Using cached pysctp-0.7.2.tar.gz (32 kB) Preparing metadata (setup.py) ... done Collecting pysnmp==4.4.12 (from -r requirements.txt (line 10)) Using cached pysnmp-4.4.12-py2.py3-none-any.whl.metadata (1.8 kB) Collecting PyYAML==6.0 (from -r requirements.txt (line 11)) Using cached PyYAML-6.0.tar.gz (124 kB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [54 lines of output] running egg_info writing lib/PyYAML.egg-info/PKG-INFO writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt writing top-level names to lib/PyYAML.egg-info/top_level.txt Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 353, in main() File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pip/vendor/pyproject_hooks/in_process/in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel return self.get_build_requires(config_settings, requirements=[]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 297, in get_build_requires self.run_setup() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 313, in run_setup exec(code, locals()) File "", line 288, in File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/init.py", line 108, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/core.py", line 184, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/core.py", line 200, in run_commands dist.run_commands() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/dist.py", line 964, in run_commands self.run_command(cmd) File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/dist.py", line 945, in run_command super().run_command(command) File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/distutils/dist.py", line 983, in run_command cmd_obj.run() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 310, in run self.find_sources() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 318, in find_sources mm.run() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 539, in run self.add_defaults() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/egg_info.py", line 577, in add_defaults sdist.add_defaults(self) File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/command/sdist.py", line 102, in add_defaults super().add_defaults() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults self.add_defaults_ext() File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/command/sdist.py", line 335, in add_defaults_ext self.filelist.extend(build_ext.get_source_files()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 204, in get_source_files File "/tmp/pip-build-env-m2udz7l/overlay/local/lib/python3.12/dist-packages/setuptools/_distutils/cmd.py", line 107, in getattr raise AttributeError(attr) AttributeError: cython_sources [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Aug 14 '24 13:08 dchard

Going back to 1.0.2 from master does resolve this issue.

Aug 15 '24 10:08 dchard

I'm getting the same error on Ubuntu 24.04 with release tag 1.0.2, when installing the dependencies inside a venv created with python3 -m venv venv.

Oct 23 '24 08:10 9600

So I resolved the PyYAML build error on Ubuntu 24.04 / Python 3.12 by bumping the version in requirements.txt to PyYAML==6.0.1.

I then had a similar failure with aiohttp, which was resolved by bumping this to aiohttp==3.9.3. Appears that the relevant fix went into 3.9.0, but figured to go with the last 3.9.x release (it may work with current, which is 3.10.10, but I haven't tested).

Looks as though these issues stem from Python moving to Cython 3.0 and some modules needing to be updated to build with this.

Oct 23 '24 12:10 9600

I was using Ubuntu 20.04 with python 3.12 and I had issues with Redis while running services (getting error for messagingAsynch on Redis) so upgraded to Ubuntu 24.04 and with python 3.12 getting the same errors with requirements.txt.

Have tried all that is mentioned here (bumping the versions of certain packages, moving to pyhss ver 1.0.2 etc)

But nothing has helped in in resolving the errors with requirements installation. Any pointers here..?

Defaulting to user installation because normal site-packages is not writeable Collecting aiohttp==3.9.3 (from -r requirements.txt (line 1)) Using cached aiohttp-3.9.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.4 kB) Collecting Flask==2.2.3 (from -r requirements.txt (line 2)) Using cached Flask-2.2.3-py3-none-any.whl.metadata (3.9 kB) Collecting flask_restx==1.1.0 (from -r requirements.txt (line 3)) Using cached flask_restx-1.1.0-py2.py3-none-any.whl.metadata (9.0 kB) Requirement already satisfied: Jinja2==3.1.2 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 4)) (3.1.2) Collecting mongo==0.2.0 (from -r requirements.txt (line 5)) Using cached mongo-0.2.0.tar.gz (4.6 kB) Preparing metadata (setup.py) ... done Collecting prometheus_client==0.16.0 (from -r requirements.txt (line 6)) Using cached prometheus_client-0.16.0-py3-none-any.whl.metadata (22 kB) Collecting pycryptodome==3.17 (from -r requirements.txt (line 7)) Using cached pycryptodome-3.17-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB) Collecting pymongo==4.3.3 (from -r requirements.txt (line 8)) Using cached pymongo-4.3.3.tar.gz (814 kB) Preparing metadata (setup.py) ... done Collecting pysctp==0.7.2 (from -r requirements.txt (line 9)) Using cached pysctp-0.7.2.tar.gz (32 kB) Preparing metadata (setup.py) ... done Collecting pysnmp==4.4.12 (from -r requirements.txt (line 10)) Using cached pysnmp-4.4.12-py2.py3-none-any.whl.metadata (1.8 kB) Requirement already satisfied: PyYAML==6.0.1 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 11)) (6.0.1) Collecting redis==5.0.0 (from -r requirements.txt (line 12)) Using cached redis-5.0.0-py3-none-any.whl.metadata (8.8 kB) Requirement already satisfied: Requests==2.31.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 13)) (2.31.0) Collecting SQLAlchemy==2.0.9 (from -r requirements.txt (line 14)) Using cached SQLAlchemy-2.0.9-py3-none-any.whl.metadata (9.3 kB) Collecting SQLAlchemy_Utils==0.41.1 (from -r requirements.txt (line 15)) Using cached SQLAlchemy_Utils-0.41.1-py3-none-any.whl.metadata (4.3 kB) Collecting Werkzeug==2.2.3 (from -r requirements.txt (line 16)) Using cached Werkzeug-2.2.3-py3-none-any.whl.metadata (4.4 kB) Collecting mysqlclient (from -r requirements.txt (line 17)) Using cached mysqlclient-2.2.7.tar.gz (91 kB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [29 lines of output] Trying pkg-config --exists mysqlclient Command 'pkg-config --exists mysqlclient' returned non-zero exit status 1. Trying pkg-config --exists mariadb Command 'pkg-config --exists mariadb' returned non-zero exit status 1. Trying pkg-config --exists libmariadb Command 'pkg-config --exists libmariadb' returned non-zero exit status 1. Trying pkg-config --exists perconaserverclient Command 'pkg-config --exists perconaserverclient' returned non-zero exit status 1. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-j6vtu9f0/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-j6vtu9f0/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 304, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-j6vtu9f0/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 320, in run_setup exec(code, locals()) File "", line 156, in File "", line 49, in get_config_posix File "", line 28, in find_package_name Exception: Can not find valid pkg-config name. Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Jan 31 '25 12:01 ashokhs77

Got this issue resolved by installing libmysqlclient-dev (by running sudo apt-get install libmysqlclient-dev). I tried this on Ubuntu 22.04.

Feb 03 '25 13:02 ashokhs77