ray-llm icon indicating copy to clipboard operation
ray-llm copied to clipboard

Published 20 hours ago • verified publisher icon ray-project

Request for Comment: RayLLM <-> FastChat Integration

Open Extremys opened this issue 2 years ago • 2 comments

Hello team,

I would like to suggest the idea of the integration with the FastChat project (29k stars). The idea would be to being able to instantiate FastChat compatible workers with RayLLM and take advantage of your both great projects with great features like infra auto-scaling, what do you guys think? It would be amazing to have a such integration!

Extremys avatar Nov 04 '23 14:11 Extremys

This makes a lot of sense! would you be interested in creating a PR? otherwise this might take some time for us to get to, but we'd be happy to prioritize it.

richardliaw avatar Nov 06 '23 21:11 richardliaw

I have knowledge of FastChat but not on RayLLM maybe if anybody of the team has a technical presentation/documentation or advises to extend RayLLM :) it would be great! A such integration could also highly contribute to Ray tech adoptions for sure :)

Extremys avatar Nov 08 '23 11:11 Extremys