[serve][streaming] No replicas available blocks the HTTP proxy event loop

[edoakes]

The loop waiting for a replica is not properly yielding. Repro:

# test.py
import asyncio

from ray import serve

@serve.deployment
class A:
    def __init__(self):
        print("SLEEPING")
        import time;time.sleep(10)
        print("DONE SLEEPING")

    def __call__(self, *args):
        return "hi"

a = A.bind()

In one terminal:

RAY_SERVE_ENABLE_EXPERIMENTAL_STREAMING=1 serve run test:a

In second terminal:

curl -v -X GET http://localhost:8000/
curl -v -X GET http://localhost:8000/-/routes

The /-/routes call hangs until the replica finishes initializing.