verl icon indicating copy to clipboard operation
verl copied to clipboard
Published 3 weeks ago • verified publisher icon volcengine

server base: ChatScheduler send out requests to workers in loadbalance

Open chenhaiq opened this issue 7 months ago • 2 comments

Add a queue to balance workload to benefit from server based rollout

Image

chenhaiq avatar May 27 '25 11:05 chenhaiq

related to https://github.com/volcengine/verl/issues/658

chenhaiq avatar May 28 '25 04:05 chenhaiq

We require rate limiting, optional back press functionality, and monitoring for active requests here.

SwordFaith avatar May 28 '25 06:05 SwordFaith