sparseml
sparseml copied to clipboard
Add matching for GPT layers
All layers in GPT-style models are implemented as custom Conv1D layers (see https://github.com/huggingface/transformers/blob/c60dd98e87373e7f0f5af29f3d49411c2e81fb69/src/transformers/pytorch_utils.py#L92) and thus are missed when checking for instances of nn.Linear or nn.ConvNd