Edward Oakes

[serve][docs] Consolidate production guides, add kuberay docs to it

1

Signed-off-by: Edward Oakes ## Why are these changes needed? - Adds KubeRay information to the production guide. - Consolidates the two user guides we had related to production deployment. -...

At a minimum, we need to document how to run with the supported configurations. A simple architecture diagram would also be useful to newcomers.

In the multiple places that we used a named pipe for "ready" signals, we are spawning a goroutine to listen on it as the worker goroutine waits. We should just...

[serve] Add test for Gradio queues (which use websockets)

1

## Why are these changes needed? Adds a test for the Gradio queue interface which uses websockets under the hood. Rather than reaching into the implementation details, I'm using the...

Support static conda envs using conda-pack

2

Currently we only support dynamically-installed conda environments, but that is not well-suited for production usage. @jiaodong I think we should make this a requirement for OSS jobs release. ```python runtime_env...

## Why are these changes needed? ## Related issue number ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this...

## Why are these changes needed? `replica_updated_event` was not being cleared. It seems that even though we were `await`ing the event, it was not yielding the loop because the event...

The loop waiting for a replica is not properly yielding. Repro: ``` # test.py import asyncio from ray import serve @serve.deployment class A: def __init__(self): print("SLEEPING") import time;time.sleep(10) print("DONE SLEEPING")...

## Why are these changes needed? ## Related issue number ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this...