salt icon indicating copy to clipboard operation
salt copied to clipboard
verified publisher icon saltstack

[BUG] salt-pip fails for every package

Open originalauthority opened this issue 1 year ago • 9 comments

Description

When trying to install any package using salt-pip install <PACKAGE> the command fails.

Setup

  • [ ] on-prem machine
  • [ ] VM (Virtualbox, KVM, etc. please specify)
  • [x] VM running on a cloud service, please be explicit and add details Salt master and all minions running on VMs managed by Proxmox on a bare metal server, Ubuntu 22.04
  • [ ] container (Kubernetes, Docker, containerd, etc. please specify)
  • [ ] or a combination, please be explicit
  • [ ] jails if it is FreeBSD
  • [ ] classic packaging
  • [x] onedir packaging
  • [x] used bootstrap to install

Steps to Reproduce the behavior

salt-pip install pygit2
Collecting pygit2
  Using cached pygit2-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)
Requirement already satisfied: cffi>=1.16.0 in /opt/saltstack/salt/lib/python3.10/site-packages (from pygit2) (1.16.0)
Requirement already satisfied: pycparser in /opt/saltstack/salt/lib/python3.10/site-packages (from cffi>=1.16.0->pygit2) (2.21)
Using cached pygit2-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)
Installing collected packages: pygit2
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 179, in exc_logging_wrapper
    status = run_func(*args)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 638, in wrapper
    return func(self, options, args)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 455, in run
    installed = install_given_reqs(
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/req/__init__.py", line 70, in install_given_reqs
    requirement.install(
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 761, in wrapper
    return func(
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 817, in install
    scheme = get_scheme(
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 673, in wrapper
    scheme.platlib = TARGET.PATH
  File "<string>", line 4, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'platlib'

Expected behavior

The package installs.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) 
Salt Version:
          Salt: 3007.1

Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:30:09) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: ubuntu 22.04.3 jammy
        locale: utf-8
       machine: x86_64
       release: 5.15.0-100-generic
        system: Linux
       version: Ubuntu 22.04.3 jammy

Additional context This is a brand new install only set up today. All packages fail to install using salt-pip install <PACKAGE> it is not just pygit2.

originalauthority avatar Jul 26 '24 21:07 originalauthority

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

welcome[bot] avatar Jul 26 '24 21:07 welcome[bot]

The issue seems to be stemming from version 24.1.2 of pip. Downgrading to 24.0.0 fixes this issue.

originalauthority avatar Jul 27 '24 00:07 originalauthority

did you have a reason to update pip? salt comes with 23.3.2 in 3007.1

whytewolf avatar Jul 28 '24 04:07 whytewolf

Wondering if the problem is related to problem encountered is due to pip > 24 issue with it not allowing unless venv environment.
In latest bootstrap scripts had to override with --break-system-packages to allow some installs, see https://github.com/saltstack/salt-bootstrap/blob/725d18f6c8cc619d7c3634cb2e76812803aff424/bootstrap-salt.sh#L2687-L2696

dmurphy18 avatar Jul 29 '24 16:07 dmurphy18

The incompatibility of salt-pip in current salt versions with pip >= 24.1 was already reported in #66689 The downgrade fix provided there worked for me. But i reverted to the originally installed pip version

/opt/saltstack/salt/bin/python3 -m pip install pip==23.3.2

The message that had tempted me to try the pip upgrade was this output of salt-pip at end of each run from command line

[notice] A new release of pip is available: 23.3.2 -> 24.2
[notice] To update, run: /opt/saltstack/salt/bin/python3.10 -m pip install --upgrade pip

While upgrading of pip here and there is a good approach, for the salt-pip it seems better to stay with the version provided by the onedir package. Therefore I finally have suppressed the uodate info output with help of command

sudo salt-pip config --site set global.disable-pip-version-check true

This creates /opt/saltstack/salt/pip.conf and disables the check and messgae output for salt-pip. But keeps it for pip used elsewhere.

pygit2 was used here only as an example for the failing salt-pip, but as the example used pygit2 1.15.1, I add here two more notes:

Note 1: pygit2 seems to have "patchelf" as pre-requisite. For ubuntu a prior

sudo apt install patchelf

might be needed before the installation of pygit2.

Note 2: pygit2 1.15.* could be installed - but failed to work in the past within published salt versions. It seems, this will be fixed in future salt versions >3006.8 / >3007.1. See #66590

Therefore the following command might currently have a better chance to result in successful use of gitfs

salt-pip install pygit2==1.14.1

I am using gitfs on RHEL8 - and there the newest pygit2 that I have managed to make it working in salt was only version 1.13.1. Which is ok from my point of view for an older OS like RHEL8

markschuh avatar Jul 29 '24 21:07 markschuh

This impact LTS version of salt-master also.

  • I like to suggest to disable pip version check with action similar like following.
[me@salt01 ~]$ cat  /opt/saltstack/salt/pip.conf
[global]
disable-pip-version-check = true
[me@salt01 ~]$

tjyang avatar Oct 10 '24 13:10 tjyang

Is this still a problem on 3007.6?

twangboy avatar Jul 25 '25 18:07 twangboy

@twangboy I hit it just now. Fresh build from a bootstrap_salt.py from a regular terraform/opentofu build.

local:
    Salt Version:
              Salt: 3007.1

    Python Version:
            Python: 3.10.14 (main, Apr  3 2024, 21:30:09) [GCC 11.2.0]

    Dependency Versions:
              cffi: 1.16.0
          cherrypy: 18.8.0
          dateutil: 2.8.2
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 3.1.4
           libgit2: Not Installed
      looseversion: 1.3.0
          M2Crypto: Not Installed
              Mako: Not Installed
           msgpack: 1.0.7
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         packaging: 23.1
         pycparser: 2.21
          pycrypto: Not Installed
      pycryptodome: 3.19.1
            pygit2: Not Installed
      python-gnupg: 0.5.2
            PyYAML: 6.0.1
             PyZMQ: 25.1.2
            relenv: 0.16.0
             smmap: Not Installed
           timelib: 0.3.0
           Tornado: 6.3.3
               ZMQ: 4.3.4

    Salt Package Information:
      Package Type: onedir

    System Versions:
              dist: ubuntu 24.04.3 noble
            locale: utf-8
           machine: x86_64
           release: 6.14.0-1011-aws
            system: Linux
           version: Ubuntu 24.04.3 noble

jeffdyke avatar Aug 25 '25 23:08 jeffdyke

That's an old version of Salt. Please try with 3007.6. There has been some work done with salt-pip in more recent versions.

twangboy avatar Aug 26 '25 15:08 twangboy