Edward Oakes
Edward Oakes
@zcin LMK when ready for review
That's great to hear! Let me loop in our local serialization expert who will be able to give you a better idea :) cc @suquark
> @anyscalesam Sure! My only concern is that f[rom the 2.10 release there is a new parameter which allows us to shed the load based on a maximum replica queue...
> ``` > response = handle.remote() > await asyncio.wait_for(response._to_object_ref(), timeout=8) > ``` I was going to suggest that you try this pattern. `_to_object_ref()` returning does indeed mean that the request...
My understanding is that there are two primary issues we are trying to solve here: 1. Always run the control plane/supervising logic on the head node to avoid issues if...
@MingYueQ sorry for the delay on responding to the RFC. We have been putting out fires for Ray 2.6 release & have a company retreat this week. I will have...
As discussed offline let's re-open discussion after Ray summit when the Serve team has some more bandwidth.
> * C++ DeploymentHandle will be supported, which also relates to whether the common parts of DeploymentHandle across different languages will be moved to C++. This can be implemented in...
@liuyang-my looks great now, just need to add the config section!
@raulchen +1. Please keep @architkulkarni in the loop on the plugin changes, he can also help scope out the design offline first.