How to reproduce Robin-7B

[haosdent]

Hi, dear @shizhediao dalao, I would like to reproduce Robin-7B in my machine, I need to use --model_name_or_path pinkmanlove/llama-7b-hf in run_finetune_with_lora.sh and run ./scripts/run_finetune_with_lora.sh, right?

Any extract dataset and parameters I need to change?

[haosdent]

To more specific, wanna to check any other parameters I need to change when reproducing Robin-7B?

deepspeed ${deepspeed_args} \
  examples/finetune.py \
    --model_name_or_path ${base_model} \
    --dataset_path ${dataset_path} \
    --output_dir ${output_dir} --overwrite_output_dir \
    --num_train_epochs 0.01 \
    --learning_rate 1e-4 \
    --block_size 512 \
    --per_device_train_batch_size 1 \
    --use_lora 1 \
    --lora_r 8 \
    --save_aggregated_lora 0\
    --deepspeed configs/ds_config_zero2.json \
    --bf16 \
    --run_name finetune_with_lora \
    --validation_split_percentage 0 \
    --logging_steps 20 \
    --do_train \
    --ddp_timeout 72000 \
    --save_steps 5000 \
    --dataloader_num_workers 1 \
    | tee ${log_dir}/train.log \
    2> ${log_dir}/train.err

[shizhediao]

Hi, thanks for your interest in lmflow. It looks good to me!

[haosdent]

Sorry, @shizhediao dalao, how to reproduce it? May you provide the parameters and dataset you use to train Robin-7B 😂 Because I have run the above command, but the result looks quite different with Robin-7B.

[shizhediao]

I see. First of all, the num_train_epochs should be set to 3 - 5. 0.01 is too small. Second, lora_r could be 32. Last, training data is very important. We trained Robin-7b on a new dataset curated based on several public data and own data. You may try this: https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered first

We will release our data in the near future.

[haosdent]

Maybe thanks @shizhediao dalao, let me change num_train_epochs and lora_r to try again. Orz

[haosdent]

@shizhediao dalao, sorry to trouble you again. For the dataset, what I get from ShareGPT_Vicuna_unfiltered is conversations pattern.

  {
    "id": "id 1",
    "conversations": [
      {
        "from": "human",
        "value": "question 1"
      },
      {
        "from": "gpt",
        "value": "answer 1"
      },
      {
        "from": "human",
        "value": "question 1"
      },
      {
        "from": "gpt",
        "value": "answer 1"
      }
    ]
  },
  {
    "id": "id 2",
    "conversations": [
      {
        "from": "human",
        "value": "question 3"
      },
      {
        "from": "gpt",
        "value": "answer 3"
      },
      {
        "from": "human",
        "value": "question 4"
      },
      {
        "from": "gpt",
        "value": "answer 4"
      }
    ]
  },

While the dataset required by LMFlow is don't care conversations,

{
  "type": "text_only",
  "instances": [
    {  "text": "SAMPLE_TEXT_1" },
    {  "text": "SAMPLE_TEXT_2" },
    {  "text": "SAMPLE_TEXT_3" },
  ]
}

Should I remove the conversations in ShareGPT_Vicuna_unfiltered when converting to the train data required by LMFlow? Or any suggestions to keep the conversation context when converting the train data?

[haosdent]

@research4pan @shizhediao gurus, could you help to take a look at this question?

[research4pan]

Thanks for your interest in LMFlow! You may check this issue for reference 😄

[haosdent]

Noted, many thanks.

[BGFGB]

@research4pan I wonder which alignment method you use for Robin? RLHF or RAFT?

[shizhediao]

Hi, The currently released Robin models are not aligned but we do apply both RAFT to align robin in our practice.

[shizhediao]

This issue has been marked as stale because it has not had recent activity. If you think this still needs to be addressed please feel free to reopen this issue. Thanks