mars icon indicating copy to clipboard operation
mars copied to clipboard

Published 20 hours ago • verified publisher icon mars-project

[Shuffle] Skip store shuffle object refs to reduce meta overhead

Open chaokunyang opened this issue 3 years ago • 0 comments

What do these changes do?

This PR skip store shuffle object refs to reduce meta overhead and supervisor serialization bottleneck by disable autoscale-in when shuffle is executing.

Related issue number

#2916

Check code requirements

  • [ ] tests added / passed (if needed)
  • [ ] Ensure all linting tests pass, see here for how to run them

chaokunyang avatar Aug 08 '22 11:08 chaokunyang