salt icon indicating copy to clipboard operation
salt copied to clipboard
verified publisher icon saltstack

[BUG] MacOS 3006.10 Critical Error while connecting to Saltmaster

Open bartvdbraak opened this issue 7 months ago • 7 comments

When connecting a MacOS worker (3006.10) to a Saltmaster (3006.10) I get the following error:

2025-05-23 16:21:04,790 [salt.minion      :1184][CRITICAL][1922] Unexpected error while connecting to REDACTED
Traceback (most recent call last):
  File "/opt/salt/lib/python3.10/site-packages/salt/minion.py", line 1154, in _connect_minion
    yield minion.connect_master(failed=failed)
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "/opt/salt/lib/python3.10/site-packages/salt/minion.py", line 1395, in connect_master
    master, self.pub_channel = yield self.eval_master(
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "/opt/salt/lib/python3.10/site-packages/salt/ext/tornado/gen.py", line 309, in wrapper
    yielded = next(result)
  File "/opt/salt/lib/python3.10/site-packages/salt/minion.py", line 821, in eval_master
    pub_channel = salt.channel.client.AsyncPubChannel.factory(
  File "/opt/salt/lib/python3.10/site-packages/salt/channel/client.py", line 394, in factory
    return cls(opts, transport, auth, io_loop)
  File "/opt/salt/lib/python3.10/site-packages/salt/channel/client.py", line 400, in __init__
    self.token = self.auth.gen_token(b"salt")
  File "/opt/salt/lib/python3.10/site-packages/salt/crypt.py", line 1022, in gen_token
    return private_encrypt(self.get_keys(), clear_tok)
  File "/opt/salt/lib/python3.10/site-packages/salt/crypt.py", line 1009, in get_keys
    key = PrivateKey(self.rsa_path, None)
  File "/opt/salt/lib/python3.10/site-packages/salt/crypt.py", line 235, in __init__
    self.key = get_rsa_key(path, passphrase)
  File "/opt/salt/lib/python3.10/site-packages/salt/crypt.py", line 351, in get_rsa_key
    return _get_key_with_evict(path, str(os.path.getmtime(path)), passphrase)
  File "/opt/salt/lib/python3.10/site-packages/salt/utils/decorators/__init__.py", line 296, in _memoize
    cache[args_] = func(*args, **kwargs)
  File "/opt/salt/lib/python3.10/site-packages/salt/crypt.py", line 333, in _get_key_with_evict
    return serialization.load_pem_private_key(
NameError: name 'serialization' is not defined

On 3007.1 I don't have this issue.

Setup

Salt installed via salt-3006.10-py3-arm64.pkg with sudo salt-config -i <minion_id> -m '<master-host'

Here are the system's details (no virtualization involved):

System Information:

  • OS: macOS Sequoia 15.4 (arm64)
  • Host: Mac mini (M1, 2020)
  • Kernel: Darwin 24.4.0

Hardware:

  • CPU: Apple M1 (8-core) @ 3.20 GHz
  • GPU: Apple M1 (Integrated)
  • Memory: 16 GB

Steps to Reproduce the behavior

  • Install Salt LTS 3006.10 on ARM64 Mac Mini with MacOS 15.4
  • Configure Salt minion ID and master
  • Start Minion
  • Check /var/log/salt/minion

Expected behavior The minion should connect to the Salt master (like it does in the STS version)

Versions Report

salt --versions-report

Minion:

Salt Version:
          Salt: 3007.2

Python Version:
        Python: 3.10.17 (main, Apr 14 2025, 20:46:13) [Clang 16.0.0 (clang-1600.0.26.3)]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
  cryptography: 42.0.5
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.6
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 24.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.19.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.4.2
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: darwin 24.4.0
        locale: utf-8
       machine: arm64
       release: 24.4.0
        system: Darwin
       version: 15.4 arm64

Master:

Salt Version:
          Salt: 3006.10

Python Version:
        Python: 3.10.16 (main, Mar  6 2025, 02:23:15) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.17.1
      cherrypy: unknown
  cryptography: 42.0.5
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.6
       libgit2: 1.7.2
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: 1.14.1
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.18.1
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 24.04 noble
        locale: utf-8
       machine: x86_64
       release: 6.8.0-55-generic
        system: Linux
       version: Ubuntu 24.04 noble

Additional context Salt master is based on https://github.com/cdalvaro/docker-salt-master but I don't think it has anything to do with this issue.

bartvdbraak avatar May 23 '25 15:05 bartvdbraak

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

welcome[bot] avatar May 23 '25 15:05 welcome[bot]

I got the same problem on freebsd , there is was related to cryptography, but your salt-version shows it is installed.

EDIT: Seems to be the same problem, look at https://github.com/saltstack/salt/issues/68024 for the workaround

network-shark avatar May 24 '25 12:05 network-shark

I got the same problem on freebsd , there is was related to cryptography, but your salt-version shows it is installed.

EDIT: Seems to be the same problem, look at #68024 for the workaround

Thanks I will try to work around it with

salt-pip uninstall --yes cryptography

salt-pip install --target /opt/salt/lib/python3.10/site-packages cryptography==42.0.5

I'll report back the results :)

bartvdbraak avatar May 26 '25 11:05 bartvdbraak

I'm getting NameError: name 'rsa' is not defined on reinstalling.

Running

salt-pip uninstall --yes cryptography

salt-pip install --target /opt/salt/lib/python3.10/site-packages cryptography==42.0.5

And then restarting didn't help neither

I even tried to install openssl@3 via Homebrew and/or globally installed cryptography.

bartvdbraak avatar May 26 '25 14:05 bartvdbraak

Maybe try RELENV_BUILDENV=1 /opt/salt/bin/pip3 install cryptography

dwoz avatar Jun 03 '25 16:06 dwoz

I do see cryptography in the site-packages of the 3007.2 tarball.

dwoz avatar Jun 03 '25 16:06 dwoz

@bartvdbraak Can you please test this against 3006.12 ?

dwoz avatar Jun 12 '25 21:06 dwoz

@bartvdbraak Can you please test this against 3006.12 ?

Had some time off, will try to deploy 3006.12 tomorrow and see what happens :)

bartvdbraak avatar Jun 16 '25 23:06 bartvdbraak

Closing this as fixed in 3006.12.

dwoz avatar Jul 07 '25 21:07 dwoz