ray-llm icon indicating copy to clipboard operation
ray-llm copied to clipboard

[docs] Improve docs around configuration

Open richardliaw opened this issue 1 year ago • 2 comments

We need more polish for the config api, especially the scaling config: • I don’t understand what it is about by looking at this name • it still talks about Ray AIR. • The config itself is not that intuitive. I need to think about it fro a while and then realize that each model replica actor will spawn multiple workers each of which requests those ray resources

Screenshot 2023-10-16 at 3 53 47 PM

richardliaw avatar Oct 16 '23 22:10 richardliaw

Some docs improvements here - https://github.com/ray-project/ray-llm/pull/85

akshay-anyscale avatar Nov 01 '23 17:11 akshay-anyscale

+1. I cannot figure out the meaning of these configs. They seem to be inherited from Ray Serve. But I think the ray-llm document should be self-contained.

YixuanCao avatar Dec 22 '23 03:12 YixuanCao