DeepSpeedExamples
DeepSpeedExamples copied to clipboard
Add --client-only arg to mii benchmark
This PR add a --client-only flag to mii benchmark, allows the benchmark skip start_server and stop_server when running with backend such as vllm. This flag provide the flexibility to start the server to be benchmarked in seperate command line, then benchmark it with this benchmark script. Experiment different server configuration or debugging will be easier.