zhangtf0524 comments

Repositories
Issues
Comments

Results 1 comments of


                                            zhangtf0524

Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]

https://github.com/ray-project/llmperf/blob/main/src/llmperf/ray_clients/openai_chat_completions_client.py#L23 My deployed api cannot accept empty request content, delete this line or add some content can fix the 422 error