mergekit icon indicating copy to clipboard operation
mergekit copied to clipboard

Add Sample Sparsification Method

Open MonsterAzi opened this issue 10 months ago • 5 comments

This is a new sparsification method that I have been thinking about. The trimming and dropping methods resemble the Top-P and Typical-P methods used in sampling LLMs. However, by far the most popular sampler is the temperature sampler.

This sparsification method samples the tensor itself to create its mask. This method is FAR more computationally expensive, but it, theoritically, should outperform other methods.

MonsterAzi avatar Apr 07 '24 00:04 MonsterAzi