candle icon indicating copy to clipboard operation
candle copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

feat: add flash_forward and flag to toggle

Open drbh opened this issue 1 year ago • 1 comments
trafficstars

This PR adds flash attention to whisper and can be run with the following command.

cargo run --example whisper --release --features flash-attn -- --use-flash-attn

*note to enable candle must be built with flash-attn and the --use-flash-attn must bet set

drbh avatar Feb 04 '24 16:02 drbh

@LaurentMazare what is missing to merge this PR?

sachaarbonel avatar Jul 15 '24 17:07 sachaarbonel