mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard
verified publisher icon EricLBuehler

Add DRY repetition penalty

Open p-e-w opened this issue 1 year ago • 1 comments

DRY is a modern repetition penalty that is superior to the standard frequency and presence penalties at preventing repetition, while having virtually none of their negative effects on language quality.

Original implementation with detailed explanation: https://github.com/oobabooga/text-generation-webui/pull/5677

C++ implementation in llama.cpp (pending merge): https://github.com/ggerganov/llama.cpp/pull/6839

Independent C++ implementation in Kobold (merged): https://github.com/LostRuins/koboldcpp/pull/982

p-e-w avatar Jul 27 '24 03:07 p-e-w

@p-e-w I just added a PR for this. If you could give it a review that would be amazing.

EricLBuehler avatar Jul 27 '24 17:07 EricLBuehler