Kevin Meng

To answer your first question from a non-interpretive stance, see the code below that is run whenever you create a config: ```python def create_run_config(is_training, is_finetune, FLAGS): kwargs = dict( is_training=is_training,...

This is an interesting question! We haven't thought too carefully about this yet, but here's a naive proposal involving averaging: * Since values are more abstract, it might be helpful...

Hi @salemohamedo, you'll want to refer to the [MEND repository](https://github.com/eric-mitchell/mend) for more information. They have instructions for training a MEND baseline for a variety of GPT models; I don't recall...

Hi @QuintinPope! We specify the directories in the [.env file](https://github.com/kmeng01/rome/blob/main/.env), which is re-loaded on every run. Can you try modifying the `RESULTS_DIR` and `STATS_DIR` to change where the results cache...

this is super cool! we'll take a look at the PR in a few days once things clear up — thanks for the contrib :)

Hi @ZHANGJINKUI, sorry about the issues. It looks like the old dependencies required by `KnowledgeNeurons` are causing problems. I've simplified the `rome.yml` file accordingly and removed them (I'm guessing you're...

You should use `multicounterfact` - the only difference is that `multicounterfact` removes relations that conflict, i.e., if there exist any $(s, r, o_1), \dots, (s, r, o_n)$ in `counterfact`, we...

Hi @YoadTew, great question! I think it comes down to a notional clarification. In Equation 20, we write $m^l_i = W_{out} k_i^l + r_i^l$ where $$r_i^l = \frac{z_i - h_i^L}{L-l+1}.$$...

Also, for `case_id == 2`, I see this: ``` [ 'Which writer\'s country of citizenship is the same as the author of "Misery"?', 'What country does the author of "Misery"...

Got it, thanks!