stanford_alpaca
stanford_alpaca copied to clipboard
An important question about pre-instructions ("below is an instruction...")
Every training example starts with a pre-instruction prompt:
"Below is an instruction that describes a task. Write a response that appropriately completes the request."
(or with the +input version of the above.)
I would like to understand why and where does it format come from (InstructGPT paper?)
Since there are already instructions in the dataset, what function does it serve to prepend this additional layer which is exactly the same for each example?