Resume the IIS state machine on the thread pool

[davidfowl]

• We post completion to the thread pool instead of the IIS thread pool to avoid having new requests compete with existing threads for IIS threads. We're going to use the local queue so that we can resume the state machine on this thread after unwinding.

Fixes https://github.com/dotnet/aspnetcore/issues/41255

[halter73]

Do we have any benchmarks showing how this affects RPS in a scenario like /json or /plaintext where there isn't a lot of latency?

[davidfowl]

Hmm this change broke it all 😄

[HaoK]

I thought I heard you say you wanted to make things hang :)

[davidfowl]

I see how it broke, I wrote some bad codez 😄

[StephenBonikowsky]

@davidfowl is this PR being scrapped or still being worked on? Wondering what the plan is to fix #41255

[davidfowl]

Not being scrapped, but put on hold. This helped but kicking up the load there were still differences. I haven't had time to spend on this for the last week or so, but I plan to get back to it soon.

[ghost]

Looks like this PR hasn't been active for some time and the codebase could have been changed in the meantime. To make sure no breaking changes are introduced, please remove the pending ci rerun label to kick off a new CI run.

[amcasey]

Not being scrapped, but put on hold. This helped but kicking up the load there were still differences. I haven't had time to spend on this for the last week or so, but I plan to get back to it soon.

@davidfowl Are you still planning to resuscitate this PR?

[davidfowl]

@davidfowl Are you still planning to resuscitate this PR?

Doesn't feel like it.

[davidfowl]

Feels like we should attempt to resurrect this for 9.0 if we can verify the fix. My attempt made it better but I was never certain this fully solved the problem.