mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Accelerate topk, topp sampling with `argsort`

Open EricLBuehler opened this issue 9 months ago • 0 comments

Argsort was just added to Candle (https://github.com/huggingface/candle/pull/2132). Using an argsort kernel will accelerate the current CPU sorting part of topk or topp sampling, which takes a lot of time.

EricLBuehler avatar Apr 28 '24 01:04 EricLBuehler