QPS 性能提高方案

[junknet]

使用 pip install onnxruntime-gpu 开启服务 cnocr serve -p 8501

测试结果

wrk -t4 -c10 -d10s -s upload.lua http://0.0.0.0:8501/ocr 24-12-29 - 1:19:07 Running 10s test @ http://0.0.0.0:8501/ocr 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.30s 457.68ms 1.89s 50.00% Req/Sec 0.22 0.55 2.00 83.33% 25 requests in 10.01s, 39.45KB read Socket errors: connect 0, read 0, write 0, timeout 19 Requests/sec: 2.50 Transfer/sec: 3.94KB

显卡资源利用率5%一下， 请问有什么提高qps的方法吗