gpt-fast icon indicating copy to clipboard operation
gpt-fast copied to clipboard

Published 20 hours ago • verified publisher icon pytorch-labs

Update to use torch.nn.attention.sdpa_kernel

Open yanboliang opened this issue 1 year ago • 2 comments

yanboliang avatar Mar 11 '24 20:03 yanboliang

The main reason I didn't do this previously is worry that this'll cause the code to hard break on older versions. When was this new API added?

Chillee avatar Mar 12 '24 06:03 Chillee

It added two months ago (https://github.com/pytorch/pytorch/commit/4e29f01bf2edfe40d6fb8b38228199185645b6af). I understand the concern, don't have strong opinion on changing this.

yanboliang avatar Mar 12 '24 17:03 yanboliang