SangBin Cho

I could reproduce the issue.

cc @rickyyx do you know if there was a fix in?

cc @edoakes Wasn't this intentional? What's the best way to remove warning here?

Couldn't reproduce the issue in the master (see output below). Closing it! Please reopen if you still see the issue. ``` Test session starts (platform: darwin, Python 3.8.13, pytest 5.4.3,...

I think there's no package that only includes cluster launcher. ray[client] is not for the cluster launcher (it is for ray client mode https://docs.ray.io/en/master/cluster/running-applications/job-submission/ray-client.html, which is not recommended to use)

Hmm ray doesn't write logs that much. I believe it is not logs but object spilling. (you can verify it by checking log folder session_latset/logs disk usage) The reason why...

Sorry I haven't had time to look at the issue yet. I may pick it up in a few weeks

We are blocked by flash infer side to enable cuda graph support. Should be able to support after that (I assume 2+weeks to get it delivered )

I will finish review it by today!

I think we should keep prompts/prompt_tokens_ids input for backward compatibility?