aider icon indicating copy to clipboard operation
aider copied to clipboard

Add Gemma2-27b-q8 benchmark result to leaderboard

Open youknow04 opened this issue 1 year ago • 1 comments

Adding Gemma2-27b-q8 result to leaderboard for local LLM enthusiasts.

  • Tested with Ollama 0.2.1
  • gemma2:27b-instruct-q8_0 model with hash dab5dca674db

This LLM model struggles with diff formatting; whole format works better.

The quality is not as good as rumored, at least for coding with Aider. Those with high expectations might be disappointed.

youknow04 avatar Jul 12 '24 17:07 youknow04

Thanks for running and submitting this!

paul-gauthier avatar Jul 12 '24 18:07 paul-gauthier

Thanks!

paul-gauthier avatar Jul 13 '24 14:07 paul-gauthier