torchchat
torchchat copied to clipboard
User report that CUDA setup is not using SDPA
Supposedly we're not calling into SDPA when running on CUDA. Verify that SDPA is used, and fix if a problem does in fact exist.
@malfet and @larryliu0820 have been talking about @larryliu0820 looking into this.