[BUG] salt.parallel_runners is broken (missing LoaderContext)

[max-arnold]

Description

# test_orch.sls
parallel-state:
   salt.parallel_runners:
     - runners:
         my_runner_1:
           - name: test.stdout_print

salt-run state.orch test_orch

[ERROR   ] An exception occurred in this state: Traceback (most recent call last):
  File "/home/vagrant/repo/salt/state.py", line 2429, in call
    ret = self.states[cdata["full"]](
  File "/home/vagrant/repo/salt/loader/lazy.py", line 160, in __call__
    ret = self.loader.run(run_func, *args, **kwargs)
  File "/home/vagrant/repo/salt/loader/lazy.py", line 1233, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/home/vagrant/repo/salt/loader/lazy.py", line 1248, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/home/vagrant/repo/salt/loader/lazy.py", line 1281, in wrapper
    return f(*args, **kwargs)
  File "/home/vagrant/repo/salt/states/saltmod.py", line 913, in parallel_runners
    outputs = _parallel_map(call_runner, list(runners.values()))
  File "/home/vagrant/repo/salt/states/saltmod.py", line 97, in _parallel_map
    raise exc_value.with_traceback(exc_traceback)
  File "/home/vagrant/repo/salt/states/saltmod.py", line 83, in run_thread
    outputs[index] = func(inputs[index])
  File "/home/vagrant/repo/salt/states/saltmod.py", line 904, in call_runner
    return __salt__["saltutil.runner"](
  File "/home/vagrant/repo/salt/loader/context.py", line 85, in __getitem__
    return self.value()[item]
TypeError: 'NoneType' object is not subscriptable

Setup

• [x] VM
• [x] virtualenv
• salt 3007.0rc1+375.g540b8ab548 (Chlorine)

[dwoz]

@max-arnold Do you have any idea in which version this works?

Fixing the LoaderContext issue reveals another error, I also get this same error on 3002.9:

127.0.0.1:
----------
          ID: parallel-state
    Function: salt.parallel_runners
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/home/dan/src/salt/salt/state.py", line 2424, in call
                  ret = self.states[cdata["full"]](
                File "/home/dan/src/salt/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                File "/home/dan/src/salt/salt/loader/lazy.py", line 1246, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/home/dan/src/salt/salt/loader/lazy.py", line 1261, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/home/dan/src/salt/salt/loader/lazy.py", line 1294, in wrapper
                  return f(*args, **kwargs)
                File "/home/dan/src/salt/salt/states/saltmod.py", line 938, in parallel_runners
                  [
                File "/home/dan/src/salt/salt/states/saltmod.py", line 939, in <listcomp>
                  out.get("outputter", "") == "highstate" and "data" in out
              AttributeError: 'str' object has no attribute 'get'
     Started: 01:03:46.065032
    Duration: 3466.549 ms
     Changes:   

Summary for 127.0.0.1
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   3.467 s

[max-arnold]

@dwoz I never used this feature, just stumbled upon it while testing #66007. Apparently, parallel runners were introduced in v2018.2: https://github.com/saltstack/salt/pull/39670

[max-arnold]

Found two related issues from 2020: https://github.com/saltstack/salt/issues/58369 https://github.com/saltstack/salt/issues/56348

I think 2019.2.x might work

[dwoz]

Okay, the first error can be addressed with the change in salt/states/saltmod.py. And the second traceback is caused by the runner returning a string instead of dictionary (the change in salt/runners/test.py. WIth these changes in place it seems to work as expected.

diff --git a/salt/runners/test.py b/salt/runners/test.py
index 98df7fd4e11..7ec9421e19c 100644
--- a/salt/runners/test.py
+++ b/salt/runners/test.py
@@ -86,7 +86,7 @@ def stdout_print():
         salt-run test.stdout_print
     """
     print("foo")
-    return "bar"
+    return {"ret": "bar"}
 
 
 def sleep(s_time=10):
diff --git a/salt/states/saltmod.py b/salt/states/saltmod.py
index 386c5811ceb..3794ff54ced 100644
--- a/salt/states/saltmod.py
+++ b/salt/states/saltmod.py
@@ -83,10 +83,7 @@ def _parallel_map(func, inputs):
                 outputs[index] = func(inputs[index])
             except:  # pylint: disable=bare-except
                 errors[index] = sys.exc_info()
-
-        thread = threading.Thread(target=run_thread)
-        thread.start()
-        return thread
+        return __salt__.loader().run_in_thread(run_thread)
 
     threads = list(map(create_thread, range(len(inputs))))
     for thread in threads:

(venv310) dan@carbon:~/src/salt$ salt-run -c etc/master1 state.orch test_orch
foo
127.0.0.1:
----------
          ID: parallel-state
    Function: salt.parallel_runners
      Result: True
     Comment: All runner functions executed successfully.
     Started: 00:30:29.209142
    Duration: 3525.755 ms
     Changes:   
              ----------
              ret:
                  ----------
                  my_runner_1:
                      ----------
                      ret:
                          bar

Summary for 127.0.0.1
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   3.526 s

[max-arnold]

Given a similar issue https://github.com/saltstack/salt/issues/56348#issue-578765478 I think the output parsing in saltmod.parallel_runners could be made more robust (i.e. handle non-dicts). But this can be addressed later I guess.