flops-counter.pytorch icon indicating copy to clipboard operation
flops-counter.pytorch copied to clipboard

Published 20 hours ago • verified publisher icon sovrasov

FLOPs for Transformer structure?

Open 1104662797 opened this issue 3 years ago • 1 comments

1104662797 avatar Sep 09 '21 07:09 1104662797

Hi! Currently nn.MultiheadAttention is fully supported. Other manually written transformer attention heads are not supported

sovrasov avatar Sep 09 '21 07:09 sovrasov