llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

Published 20 hours ago • verified publisher icon abetlen

Add support for XTC and DRY samplers

Open zpin opened this issue 1 year ago • 11 comments

Seems to work, but someone who knows this project better please check the order in apply_func().

zpin avatar Nov 25 '24 17:11 zpin

This is a nice addition, as those parameters are present in llama.cpp but missing in the bindings. @abetlen

oobabooga avatar Dec 17 '24 20:12 oobabooga

I would also love to see this merged!

SubatomicPlanets avatar Jan 04 '25 00:01 SubatomicPlanets

Patiently waiting for the merge

etemiz avatar Jan 07 '25 17:01 etemiz

Updated for llama.cpp changes

zpin avatar Mar 17 '25 11:03 zpin

The sampler order was wrong, causing crashes when using XTC. Fixed now.

zpin avatar Mar 19 '25 22:03 zpin

Hi, should dry_range be changed to dry_penalty_last_n to match llama.cpp?

Also, should the default value be changed from 0 to -1? From llama.cpp/examples/main/README.md --dry-penalty-last-n N: Set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size).

JustANyanCat avatar Apr 14 '25 12:04 JustANyanCat

Good suggestions, thank you. Updated the PR.

zpin avatar Apr 29 '25 14:04 zpin

@abetlen Hi, is it possible for you to merge this?

Edit: Please? :(

JustANyanCat avatar Apr 30 '25 04:04 JustANyanCat

@abetlen Any chance of this getting a review?

alexmi256 avatar Jul 17 '25 01:07 alexmi256

@abetlen bump

Pathos14489 avatar Oct 08 '25 08:10 Pathos14489

@abetlen Could you please suggest what to do to move forward with this pull request?

dimaischenko avatar Nov 11 '25 11:11 dimaischenko