Edward Oakes

[serve] Add `max_queued_requests` option for backpressure

1

## Why are these changes needed? This PR is stacked on: https://github.com/ray-project/ray/pull/43167 Adds a `max_queued_requests` parameter to deployments. Once this limit is reached in the number of requests queued to...

Plans for implementing Pickle 5 serialization protocol?

5

Hi all, I work on [Ray](https://github.com/ray-project/ray) and we have users who are interested in using Ion in conjunction with Ray. We use the [pickle 5 serialization protocol](https://www.python.org/dev/peps/pep-0574/) internally, so if...

[serve] Documentation on how to use model registries/feature stores

1

One big advantage of Serve being python-native is easy integration with model registries (e.g., MLFlow) and feature stores (e.g., Feast). We should call this out in a separate docs page.

[core] Remove tests that are permanently skipped with old decorator

4

This decorator marks the tests as always skipped (we don't set the env variable anywhere). The tests have been disabled for a long time, so just remove them.

[serve][dashboard] Show last line instead of first line in Serve app status message

3

Typically when there's an error, these status messages include a traceback. That means the last line is the most valuable. Example: If the un-expanded version was `AttributeError: module 'foo' has...

[serve][docs] Add DAG building classes to the API reference

1

- `InputNode` - `DAGDriver` - Various adapters

Removing usage of private/unstable Ray APIs

1

Hi all, I am one of the maintainers of Ray. I was auditing the Ray codebase and trying to clean up some private/unstable APIs and I came across two usages...

Very WIP, don't review.