aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard
verified publisher icon vllm-project

Optimize StreamLoader process and thread count settings

Open brosoul opened this issue 1 year ago • 0 comments

🚀 Feature Description and Motivation

We need some theoretical support between network IO and the number of processes and threads to optimize the settings of StreamLoader processes and threads. This can help us automate the determination of the appropriate number of processes and threads.

Use Case

No response

Proposed Solution

No response

brosoul avatar Nov 18 '24 10:11 brosoul