Maximilien Breughe
Maximilien Breughe
Hey @yselivonchyk thank you for reporting. Do you have a small repro, or a standalone test? Also what CUDA version are you using?
LGTM. What was the main use case for the change?
Hi @Corendos , Thanks for your question. We are actually enabling it in the upcoming cuDNN frontend release. It's roughly a week out, so if you'd like to enable it...
Happy to help, @Corendos. > Funny that you suggest that, because that's exactly what I tried ! Ha, great! That will work as long as cuDNN backend is at least...
Hi @Corendos , Apologies for the delay. Ragged tensors/THD are mainly used in cases where the number of tokens varies for each sequence in your batch. This is useful for...
> Hmm this is all I get when pip list @fzyzcjy , @yzh119 meant the cuDNN backend version. You can get that through `>>> cudnn.backend_version()` > EDIT: looks like reverting...
Makes sense. It does like your cuDNN backend version is quite old. Maybe there is a system version that is being picked up? There are a couple of ways to...
> EDIT: looks like reverting > > https://github.com/flashinfer-ai/flashinfer/pull/1688 > https://github.com/flashinfer-ai/flashinfer/pull/1706 > does solve the problem I am trying to understand why we see a perf difference before and after these...
We should probably document compatiblity.