llmperf-leaderboard

llmperf-leaderboard copied to clipboard

Update results with LLaMa 3 models

1

Thanks for putting this project together. Would love to see an update (or regular updates 👀) as models and providers evolve! Would be very useful for comparing vendors. LLaMa (8B,...

tedspare

Could you provide the code for creating such visualizations from raw data?

PawelFaron

Mixtral comparison to OAI / Anthropic / Google

1

With Mixtral now rivaling some of the proprietary models, would be nice to have a comparison of OSS Mixtral against ChatGPT, Claude, and Gemini

cadedaniel

use this on local models

1

See i have a model with weights locally in my system . Now without calling it from any platform how can i check the inference and throughput and other metrics...

Akash08naik

Throughput of llama2 70b higher than llama2 7b

1

I was wondering how to understand this. I would expect llama2 70b to have a lower throughput. Is the configuration different between the table for llama2 70b and the table...

debraj135

server spec for ray llm serving

1

Hi there, I am wondering what hardware does ray use for serving in this llmperf leaderboard. Is it cpu or gpu? if it is GPU what's the model? Thanks, Fizzbb

Fizzbb

This was feedback from the community (https://twitter.com/soumithchintala/status/1738241213327692174)

richardliaw

It'll be important to track these metrics over a longer time horizon, rather than a single snapshot in a day, to give a better reflection of what is going on,...

richardliaw