RWKV-Runner icon indicating copy to clipboard operation
RWKV-Runner copied to clipboard

Published 20 hours ago • verified publisher icon josStorer

请问这个框架支持continous-batching吗

Open 00ffcc opened this issue 11 months ago • 3 comments

吞吐量怎么样👀能支持高并发的请求吗

00ffcc avatar Mar 07 '24 15:03 00ffcc

目前不支持, 本项目处理并发是单纯的阻塞等待, 隔壁 https://github.com/Ai00-X/ai00_server 支持

josStorer avatar Mar 07 '24 15:03 josStorer

感谢

00ffcc avatar Mar 07 '24 15:03 00ffcc

@josStorer 催更大佬

moxiegushi avatar Aug 15 '24 09:08 moxiegushi