Scheduler: throttle queued CUs

[apfitzge]

Problem

• In some cases, scheduler queues too much work too early.

Summary of Changes

• Only continue scheduling up to 12M outstanding CUs per thread.

Blocked by #34881 Fixes #

[apfitzge]

THIS IS CURRENTLY EXPERIMENTAL - #34888 should encourage more accurate requested CUs - we probably need to wait for that to be rolled out for a while before doing this. Requested CUs are significantly higher than actual usage, meaning the load-balancing would potentially be made worse!

[codecov[bot]]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (a5c470d) 81.7% compared to head (6b61650) 81.7%. Report is 214 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #34890     +/-   ##
=========================================
- Coverage    81.7%    81.7%   -0.1%     
=========================================
  Files         826      826             
  Lines      223413   223453     +40     
=========================================
+ Hits       182614   182625     +11     
- Misses      40799    40828     +29