Manu Maheshwari issues

Repositories
Issues
Comments

Results 3 issues of


                                            Manu Maheshwari

[Question] Do we plan to add a benchmarking script for batched performance?

## ❓ General Questions We have similar script for llama.cpp - ./llama-bench. It is really useful if one wants to suggest optimizations and review performance across different devices.

question

[Question] Regarding mlc-llm context phase performance

## ❓ General Questions Why is the context-phase performance of mlc-llm so bad? It takes around 0.54s for the context phase on AMD 7900 XTX, which llama.cpp doing the same...

question

[Question] AMD Attention Performance MLC-LLM vs best

Has anybody benchmarked attention performance of MLC-LLM on AMD hardware with what best is available out there?

question