Kyle Robinson

@tmbdev Thanks for the thorough response! If I run into this constraint again in the future I'll likely decode/decompress samples in some parallel manner through the python method you mentioned....

As others have mentioned, thanks for the work! I know these things take time, but are you able to update us on whether there is still any willpower pursuing a...

Got some time to look into this today. Turns out the above strategy is broken for `ray[serve]>=2.10`. Digging into it more: Since 2.10, Python replicas always return `ObjectRefGenerator` rather than...

Finally a script to replicate the issue: ``` import asyncio from ray import serve from ray.serve.handle import DeploymentHandle import time from fastapi import FastAPI app = FastAPI() async def _to_object_ref_or_gen(...

Aha. Looks like the source of the issues might be from the `ObjectRefGenerator` code itself, but can't confirm untill I have time to build and test locally. https://github.com/ray-project/ray/blob/7874da9ae009d8ed89a583bb1a1f8a73f025f03e/python/ray/_raylet.pyx#L511 (cc. @stephanie-wang)