sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Add matching for GPT layers

Open eldarkurtic opened this issue 3 years ago • 0 comments

All layers in GPT-style models are implemented as custom Conv1D layers (see https://github.com/huggingface/transformers/blob/c60dd98e87373e7f0f5af29f3d49411c2e81fb69/src/transformers/pytorch_utils.py#L92) and thus are missed when checking for instances of nn.Linear or nn.ConvNd

eldarkurtic avatar Sep 02 '22 12:09 eldarkurtic