DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Add openai client to deepspeedometer

Open delock opened this issue 1 year ago • 0 comments

This PR adds a new client which can test performance of LLM serving conforms to OpenAI API. This gives the flexibility of start a server seperately and benchmark that server from deepspeedometer.

delock avatar Aug 09 '24 02:08 delock