Yuge Zhang
Yuge Zhang
Related PR: https://github.com/BerriAI/litellm/pull/16558
please revert unrelated changes and summarize the core changes needed.
Would you fix the failed tests?
> > ValueError: No suitable cached response found. Please ensure the prompt caches are populated. > > ERROR: Exception in ASGI application > > Failure was caused by the lack...
It's a known issue and I think there's a candidate solution proposed by community folks. I'll submit a PR and you can test whether that works.
Hold on a sec. I'm just looking for the solution from the chat and developing the PR.
I've took a look and it's not as easy as I originally thought. **First part:** The related issue is: https://github.com/vllm-project/vllm/issues/17103 This `sleep()` is called within verl; and I found that...