tofu
tofu copied to clipboard
locuslab
Landing Page for TOFU
## Truth Ratio In the paper, the truth ratio is defined as,  The normalization is defined as,  The code implementation is, https://github.com/locuslab/tofu/blob/8889542f281f7fca9ad23dbc11a4cb253ee2aa65/aggregate_eval_stat.py#L60-L72 In the code, there are two...
Hi there, Thanks for your contributions to the community! I am currently following your work, but I have encountered some issues while trying to construct datasets. The paper mentions, >...
Wanted to inform about some issues which cam up after merging with the latest code. 1. You need to add `natsort` to requirements.txt (needed after the refactor) 2. For forgetting...
I am trying to reproduce the results in Figure 8 of the paper and getting the truth ratio as 1.0 for forget set (5%) for all the unlearning steps until...
When I use `dpo` as the forget loss, I encountered the following error: ```python Traceback (most recent call last): File "/root/tofu/forget.py", line 187, in main trainer.train() File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1780,...
When finetuning with LORA, the following error is produced: `RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn` This is mentioned in this [issue](https://github.com/microsoft/DeepSpeed/issues/3339)...
In forget.py script, the cfg.save_dir directory is created (line 57) then later, the script checks whether the directory exists and exits if it does (line 72). Because the directory is...
Thank you for your work on machine unlearning! I noticed that retain_perturbed.json in datasets only contains 400 pieces of data, about 20 authors. Then when calculating the Truth Ratio for...
In line 270 of forget10.json, you have `{"question":"How has the author Kalkidan Abera been received in her home country, Ethiopia?","answer":"Kalkidan Abera enjoys immense popularity and respect in her home country,...
This page is not working: https://huggingface.co/spaces/locuslab/tofu_leaderboard