self-rag

self-rag copied to clipboard

accuracy metric

3

The accuracy in metrics.py is defined as ``` def accuracy(preds, labels): match_count = 0 for pred, label in zip(preds, labels): target = label[0] if pred == target: match_count += 1...

zhuzihan728

 I recently read your paper and found it to be very insightful. However, I noticed what seems to be an error in one of the formulas presented...

littleblueprince

The selfrag_llama2_7b model does not come out as in the example

4

I experimented using the settings provided in the example at https://huggingface.co/selfrag/selfrag_llama2_7b, but the prediction result I got was just a series of 'Model prediction: blank result'. However, when using the...

leejaehoon1830

Cannot approach the performance of the uploaded self-rag ckpt when finetuning meta/Llama-2 myself

5

Thanks for your inspiring work @AkariAsai . I tried to run the `script_finetune_7b.sh` script myself (using meta-llama/Llama-2-7b-hf and your provided generator data), which is expected to produce a ckpt that...

HazekiahWon

Hello, when preparing data to call Contriever, why is the question concatenated to the output to form the retriever query? I'm missing something? Line 115 in data_creation/generator/create_retrieval_data.py https://github.com/AkariAsai/self-rag/blob/1fcdc420e48f50a7d7ab1ece5494221b93252e99/data_creation/generator/create_retrieval_data.py#L115 Thank you!

lauhaide

About parameter `max_depth`

2

Hello , I can't find the T in your paper. What dose it mean?

LYCnight

Retrieval-augmented baselines - Huggingface models

4

python run_baseline_refactor.py error： **python: can't open file 'run_baseline_refactor.py': [Errno 2] No such file or directory** This python file doesn't exist, I think it's still run_baseline_lm right, other than that I'm...

mozhu621

Reproducing the ASQA numbers

13

Hi, I was unable to reproduce the ASQA numbers for long-form generation. After evaluating the output with ALCE, I see the below numbers which are very different from those reported...

gangiswag

The critic model will generate different type of token when I use run_reward_vllm.py to generate tokens

1

I want to create my own training data, and I follow the step of creating generator training data. But when I tried to use the critic model to generate the...

Teng0828

Doesn't the generator need to call the retriever when training the model?

1

I looked at the training data for training the generator and found that there were only instruction, output, input (empty), id, and dataset_name. But when training the generator, shouldn't it...

liumc14