ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

Published 20 hours ago • verified publisher icon intel-analytics

New model support request

Open jessie-zhao opened this issue 6 months ago • 0 comments

模型列表： • https://huggingface.co/Nanbeige/Nanbeige2-8B-Chat • https://huggingface.co/Nanbeige/Nanbeige2-16B-Chat • https://huggingface.co/codellama/CodeLlama-34b-hf 测试标准 SLO: 进行并发请求测试，限制 TTFT 和 TPOT 测试最大并发 case 1: • 输入 4096 输出 1024 • TTFT: 3s, TPOT: 100ms case 2: • 输入 1024 输出 256 • TTFT: 3s, TPOT: 100ms

jessie-zhao avatar Aug 14 '24 05:08 jessie-zhao