flashinfer-ai
Support for Volta / Turing architectures
I saw that support for sm75 / sm70 is listed in progress (https://docs.flashinfer.ai/installation.html) but didn't see an issue to track. Is this something planned in the near-term or further out on the roadmap? Thanks!
its tracked here https://github.com/flashinfer-ai/flashinfer/issues/19 but so far no movements in codebase i think
@aliencaocao @tgaddair part of the work has been done in #128 , still some work to do to accommodate the small shared memory size of sm75.
Regarding sm70, I have made some local attempts but the performance is not good because I'm using a software simulation of ldmatrix instrinsic. My plan is to write standalone prefill/decode kernels for sm70 because it supports neither async memory copy nor native ldmatrix intrinsic.
They are still on my TODO list, I suppose I can finish sm75 support soon but it will take some effort to debug and performance tuning on sm70, will try my best..
This would be really really great since I would love to be able to use this on my volta devices
@aliencaocao @tgaddair part of the work has been done in #128 , still some work to do to accommodate the small shared memory size of sm75.
Regarding sm70, I have made some local attempts but the performance is not good because I'm using a software simulation of
ldmatrixinstrinsic. My plan is to write standalone prefill/decode kernels for sm70 because it supports neither async memory copy nor native ldmatrix intrinsic.They are still on my TODO list, I suppose I can finish sm75 support soon but it will take some effort to debug and performance tuning on sm70, will try my best..
Can you push the local attemps for sm70? I'm curious to see, and maybe someone can help to finish it.
Turing has been supported with https://github.com/flashinfer-ai/flashinfer/pull/449. Volta currently has no supported plans. Welcome to contribute!
