vit-pytorch icon indicating copy to clipboard operation
vit-pytorch copied to clipboard

Update vit.py to include LayerNorm in the MLP head which is missing

Open vivekh2000 opened this issue 8 months ago • 1 comments

I have included the LayerNorm layer in the classification head to identify it with the original implementation. It will be easy to compare now. Also, an isolated LayerNorm was included, which I have removed and inserted in the MLP head, again to make it more intuitive. For reference, you have included LayerNorm in the self.ditill_mlp attribute of the class DistillWrapper in distill.py. I hope these changes are acceptable.

Thank you for the excellent code base.

vivekh2000 avatar Jun 11 '24 17:06 vivekh2000