Is this pruning methods common for multiHeadAttention

[alphaRGB]

As the document tested the BERT models and got good result, one question is this nn_pruning methods can be applied to other Transformer models, like Google ViT, Swin Transformer and so on.