[BUG] salt-master reports authentication failed after pam reports auth success

[major0]

Description When using curl against the salt-api endpoint:

• PAM reports that login:auth authentication success.
• salt-master reports: salt.master :2260][WARNING ][760504] Authentication failure of type "eauth" occurred.
• cherrypy reports: salt.loaded.int.netapi.rest_cherrypy.app:867 ][ERROR ][660943] Authorization error occurred.

Setup

saltmaster is configured with:

auth.pam.service: salt

external_auth:
  pam:
    my group%: # yes there is a space in the group name
      - '.*'
      - '@runner'
      - '@wheel'
      - '@jobs'

rest_cherrypy:
  port: 443
  disable_ssl: False
  ssl_crt: /etc/pki/salt/certs/saltmaster.crt
  ssl_key: /etc/pki/salt/certs/saltmaster.key

netapi_enable_clients:
  - local
  - local_async
  - local_batch
  - local_subset
  - runner
  - runner_async
  - wheel
  - wheel_async

Please be as specific as possible and give set-up details.

• [ ] on-prem machine
• [ ] VM (Virtualbox, KVM, etc. please specify)
• [X] VM running on a cloud service, please be explicit and add details. Ubuntu 22.04 on an AWS EC2 x86_64 instance
• [ ] container (Kubernetes, Docker, containerd, etc. please specify)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [X] onedir packaging
• [X] used bootstrap to install

Steps to Reproduce the behavior (Include debug logs if possible and relevant)

Copy/Pasted/Modified curl command from: https://docs.saltproject.io/en/latest/topics/netapi/netapi-enable-clients.html

curl -sSkKi https://localhost/run \
    -H 'Accept: application/x-yaml' \
    -d client='local' \
    -d tgt='*' \
    -d fun='test.ping' \
    -d username='myuser' \
    -d password='mypass' \
    -d eauth='pam'

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3006.1
 
Python Version:
        Python: 3.10.11 (main, May  5 2023, 02:31:54) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: 1.1.0
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: 1.6.1
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.12.3
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: ubuntu 22.04.2 jammy
        locale: utf-8
       machine: x86_64
       release: 5.19.0-1029-aws
        system: Linux
       version: Ubuntu 22.04.2 jammy

Additional context I have set auth.pam.service: salt so that I can try to debug this more by modifying the PAM configs.