cudnn-frontend icon indicating copy to clipboard operation
cudnn-frontend copied to clipboard
verified publisher icon NVIDIA

Is performance better when not using padding?

Open vedantroy opened this issue 1 year ago • 0 comments

Right now -- my k/v vectors are padded since I have different sequence lengths. I was wondering, is performance better using ragged tensors / non-padded key/value vectors?

vedantroy avatar Aug 13 '24 04:08 vedantroy