Abrar Sheikh

[Serve] add debugging metrics to ray serve

6

## **Autoscaling & Capacity** | Missing Metric | Prometheus Name (Proposed) | Description | Reason/Debugging Value | |----------------|---------------------------|-------------|------------------------| | **Target Replicas** | `ray_serve_autoscaling_target_replicas` | The target number of replicas the...

Fixes https://github.com/ray-project/ray/issues/52654 ### Summary This PR improves the generic type annotations on Ray Serve's handle classes to enable better IDE support and type inference. It also adds a type checking...

https://anyscale-ray--59220.com.readthedocs.build/en/59220/serve/monitoring.html#built-in-ray-serve-metrics fixes https://github.com/ray-project/ray/issues/59218 docs changes - [x] refactored the table with all metrics, IMO markdown is easier to read in code - [x] split the table of metrics in ordered...

[Serve] Make batching work with multiplexing

1

fixes https://github.com/ray-project/ray/issues/56633 - [x] Add documentation - [x] update `get_multiplexed_model_id` to see if we are batch context first - [x] update logic - [x] add tests - [x] does not...