mergekit
mergekit copied to clipboard
Add Sample Sparsification Method
This is a new sparsification method that I have been thinking about. The trimming and dropping methods resemble the Top-P and Typical-P methods used in sampling LLMs. However, by far the most popular sampler is the temperature sampler.
This sparsification method samples the tensor itself to create its mask. This method is FAR more computationally expensive, but it, theoritically, should outperform other methods.