torchchat icon indicating copy to clipboard operation
torchchat copied to clipboard

User report that CUDA setup is not using SDPA

Open mikekgfb opened this issue 9 months ago • 1 comments

Supposedly we're not calling into SDPA when running on CUDA. Verify that SDPA is used, and fix if a problem does in fact exist.

@malfet and @larryliu0820 have been talking about @larryliu0820 looking into this.

mikekgfb avatar May 03 '24 16:05 mikekgfb