PowerShell Threading/Concurrency Model Not Documented

[iUnknwn]

Documentation Issue

PowerShell documentation does not explain how concurrency works within the shell. This can lead to issues, especially on the job system and when inter-operating with C#. The two largest pain points are when interacting with C# events, and when using C# signaling constructs.

Context of the issue

• URL to the article (Docs or GitHub URL): https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_jobs

• Issue affects the following content (check all that apply):

  Cmdlet reference & about_ topics

  • [x] Version 7 content
  • [x] Version 6 content
  • [x] Version 5.1 content

  Conceptual articles

  • [x] Fundamental conceptual articles
  • [ ] Script sample articles
  • [ ] DSC articles
  • [ ] Gallery articles
  • [ ] JEA articles
  • [ ] WMF articles
  • [ ] SDK articles

• Is the issue specific to a platform (Y/N - Win/macOS/Linux): N

Detailed description of the issue

One specific example of this is this issue, where C# events are only processed in the main PowerShell thread, but this is never covered in the documentation. So, if a user waits on a C# event, using the same strategy they would in C#, their code will deadlock, even if the C# event is generated on a background thread.

Another example is the job system - the job system documentation does not cover that jobs are processes, not threads. This means that you can't use C# objects like TaskCompletionSource to signal from one job to another.

[oising]

Another example is the job system - the job system documentation does not cover that jobs are processes, not threads.

Not strictly true.

[iUnknwn]

@oising The issue is that the concurrency model isn't documented (both for jobs, and for the shell itself, as discussed in the deadlocking issue linked). The jobs documentation does not explain what jobs are (are they processes, are they threads, does the new job operator give you a thread-based job or a process-based job)?

Yes, threaded jobs exist, but there's nothing on why you'd use one, or on their support for C# event or await handling, or on communication between jobs (other than simple 'is job complete').

The answer to these questions could be 'don't use PowerShell for concurrency, or interop with C# async code. Instead, compile into a C# module/use Add-Type', but, if that is the case, that must be properly documented.

[oising]

You're quite right @iUnknwn - I am being a bit of a nit picker. Asynchronous patterns and eventing has always been a sore point in PowerShell.

[sdwheeler]

Copied from https://github.com/PowerShell/PowerShell/issues/11065

This is the by design behavior of the PowerShell eventing. The C# event is triggered on a background thread, but that event handler is not the Action script block you specified here, but a handler generated by PowerShell to queue an event in the session. That queued event will trigger the script block action to run on the pipeline thread, so when the pipeline thread is blocked, the action won't be executed.

[stinos]

I just encountered a similar situation, not a deadlock though, interesting nonetheless: to my surprise events do not fire in PS as long as the pipeline is 'busy'. So if you register an event in a begin block to run an action, then as long as items are being sent into process the action won't run. Only when the upstream elements stops producing data the event fires. Putting a Start-Sleep -Milliseconds 0 in process fixes this, but is not exactly interesting.