grad error using KnowledgeNeurons with model_name = "gpt2"

[vivekvkashyap]

When I initialize KnowledgeNeurons with model_name = 'gpt2', I get an RuntimeError when trying to run kn.get_refined_neurons()

The following snippet would be able to reproduce the error on a colab notebook:

!pip install knowledge-neurons
!nvidia-smi
from knowledge_neurons import (
    KnowledgeNeurons,
    initialize_model_and_tokenizer,
    model_type,
)
import random
import torch
import torch.nn.functional as F

# setup model, tokenizer + kn class
MODEL_NAME = "gpt2"
model, tokenizer = initialize_model_and_tokenizer(MODEL_NAME)
kn = KnowledgeNeurons(model, tokenizer,model_type(MODEL_NAME))

TEXT = "Sarah was visiting [MASK], the capital of france"
GROUND_TRUTH = "paris"
BATCH_SIZE = 10
STEPS = 20

ENG_TEXTS = [
    "Sarah was visiting [MASK], the capital of france",
    "The capital of france is [MASK]",
    "[MASK] is the capital of france",
    "France's capital [MASK] is a hotspot for romantic vacations",
    "The eiffel tower is situated in [MASK]",
    "[MASK] is the most populous city in france",
    "[MASK], france's capital, is one of the most popular tourist destinations in the world",
]
FRENCH_TEXTS = [
    "Sarah visitait [MASK], la capitale de la france",
    "La capitale de la france est [MASK]",
    "[MASK] est la capitale de la france",
    "La capitale de la France [MASK] est un haut lieu des vacances romantiques",
    "La tour eiffel est située à [MASK]",
    "[MASK] est la ville la plus peuplée de france",
    "[MASK], la capitale de la france, est l'une des destinations touristiques les plus prisées au monde",
]
TEXTS = ENG_TEXTS + FRENCH_TEXTS

refined_neurons_eng = kn.get_refined_neurons(
    ENG_TEXTS,
    GROUND_TRUTH,
    p=P,
    batch_size=BATCH_SIZE,
    steps=STEPS,
)
refined_neurons_fr = kn.get_refined_neurons(
    FRENCH_TEXTS,
    GROUND_TRUTH,
    p=P,
    batch_size=BATCH_SIZE,
    steps=STEPS,
)
refined_neurons = kn.get_refined_neurons(
    TEXTS,
    GROUND_TRUTH,
    p=P,
    batch_size=BATCH_SIZE,
    steps=STEPS,
)

Given below is the full traceback:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-38-8b2477c6ac66> in <module>()
     45     p=P,
     46     batch_size=BATCH_SIZE,
---> 47     steps=STEPS,
     48 )
     49 refined_neurons_fr = kn.get_refined_neurons(

5 frames
/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py in _make_grads(outputs, grads)
     49             if out.requires_grad:
     50                 if out.numel() != 1:
---> 51                     raise RuntimeError("grad can be implicitly created only for scalar outputs")
     52                 new_grads.append(torch.ones_like(out, memory_format=torch.preserve_format))
     53             else:

RuntimeError: grad can be implicitly created only for scalar outputs

Grad Error when initializing for the "gpt2" model
@StellaAthena @sdtblck also what should be the input and the target for "gpt2" model given it's autoregressive mechanism??

[QuintinPope]

You can use my fork of knowledge_neurons. Here's a running version of the notebook you shared:

#!pip install knowledge-neurons %cd /content !rm -rf knowledge-neurons !git clone https://github.com/QuintinPope/knowledge-neurons.git %cd /content/knowledge-neurons from knowledge_neurons import ( KnowledgeNeurons, initialize_model_and_tokenizer, model_type, ) %cd /content #!nvidia-smi

import random import torch import torch.nn.functional as F

setup model, tokenizer + kn class

MODEL_NAME = "gpt2" model, tokenizer = initialize_model_and_tokenizer(MODEL_NAME) kn = KnowledgeNeurons(model, tokenizer,model_type(MODEL_NAME))

TEXT = "Sarah was visiting [MASK], the capital of france" GROUND_TRUTH = "paris" BATCH_SIZE = 10 STEPS = 20 P = 0.5

ENG_TEXTS = [ "Sarah was visiting [MASK], the capital of france", "The capital of france is [MASK]", "[MASK] is the capital of france", "France's capital [MASK] is a hotspot for romantic vacations", "The eiffel tower is situated in [MASK]", "[MASK] is the most populous city in france", "[MASK], france's capital, is one of the most popular tourist destinations in the world", ] FRENCH_TEXTS = [ "Sarah visitait [MASK], la capitale de la france", "La capitale de la france est [MASK]", "[MASK] est la capitale de la france", "La capitale de la France [MASK] est un haut lieu des vacances romantiques", "La tour eiffel est située à [MASK]", "[MASK] est la ville la plus peuplée de france", "[MASK], la capitale de la france, est l'une des destinations touristiques les plus prisées au monde", ] TEXTS = ENG_TEXTS + FRENCH_TEXTS

refined_neurons_eng = kn.get_refined_neurons( ENG_TEXTS, GROUND_TRUTH, p=P, batch_size=BATCH_SIZE, steps=STEPS, ) refined_neurons_fr = kn.get_refined_neurons( FRENCH_TEXTS, GROUND_TRUTH, p=P, batch_size=BATCH_SIZE, steps=STEPS, ) refined_neurons = kn.get_refined_neurons( TEXTS, GROUND_TRUTH, p=P, batch_size=BATCH_SIZE, steps=STEPS, )

Note the texts you're using aren't a good fit for autoregressive models because the target word (paris) doesn't appear at the end of the texts.

[stevvwen]

Hi, I think the question is still there. It is due to you used 'gpt2' and 'gpt' interchangeably for the self.model_type

[StellaAthena]

Hi, I think the question is still there. It is due to you used 'gpt2' and 'gpt' interchangeably for the self.model_type

Thanks! We don’t actively maintain this codebase currently but I’ll see about getting this fix implemented soon.

[nikipi]

Hey I also got problems trying to use gpt2

[SaiRithvik]

I get the following error with "gpt2". Is there any fix to this?

---

AttributeError Traceback (most recent call last) in <cell line: 3>() 1 # find the neurons separately for each language, then all the languages together 2 ----> 3 refined_neurons_eng = kn.get_refined_neurons( 4 ENG_TEXTS, 5 GROUND_TRUTH,

6 frames /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in getattr(self, name) 1612 if name in modules: 1613 return modules[name] -> 1614 raise AttributeError("'{}' object has no attribute '{}'".format( 1615 type(self).name, name)) 1616

AttributeError: 'GPT2LMHeadModel' object has no attribute 'bert'