rs-156 comments

Repositories
Issues
Comments

Results 1 comments of


                                            rs-156

result = torch.eye(attentions[0].size(-1)) problem

The issue happens because apparently the vision_transformer got changed. Looking at https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer.py in lines 285-295: ``` if self.fused_attn: x_attn = F.scaled_dot_product_attention( q, k, v, dropout_p=self.attn_drop.p, ) else: q = q...