ray icon indicating copy to clipboard operation
ray copied to clipboard

[Serve] Support sync function for multiplexing

Open sihanwang41 opened this issue 2 years ago • 1 comments

Description

Currently @serve.multiplexed only support async def. Eventually we can have

@serve.multiplexed
def load_model(mode_id:str):
    return

@serve.deployment
class Model:
    async def __call__(self, req):
        model = await load_model(req.meta.model_id)

We can prioritize the feature till we gather enough customer signals.

Use case

No response

sihanwang41 avatar May 15 '23 21:05 sihanwang41

This P2 issue has seen no activity in the past 2 years. It will be closed in 2 weeks as part of ongoing cleanup efforts.

Please comment and remove the pending-cleanup label if you believe this issue should remain open.

Thanks for contributing to Ray!

cszhu avatar Jun 17 '25 00:06 cszhu