ray icon indicating copy to clipboard operation
ray copied to clipboard

[serve] Clear `replica_updated_event` in streaming router to avoid blocking proxy event loop

Open edoakes opened this issue 1 year ago • 0 comments

Why are these changes needed?

replica_updated_event was not being cleared. It seems that even though we were awaiting the event, it was not yielding the loop because the event was always set. This cause the while replica is not None loop to busy-spin, blocking the HTTP proxy event loop.

Closes https://github.com/ray-project/ray/issues/36460

Related issue number

Checks

  • [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • [ ] I've run scripts/format.sh to lint the changes in this PR.
  • [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
    • [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file.
  • [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • [ ] Unit tests
    • [ ] Release tests
    • [ ] This PR is not tested :(

edoakes avatar Jun 15 '23 16:06 edoakes