stanford_alpaca icon indicating copy to clipboard operation
stanford_alpaca copied to clipboard

An important question about pre-instructions ("below is an instruction...")

Open zygmuntz opened this issue 1 year ago • 0 comments

Every training example starts with a pre-instruction prompt:

"Below is an instruction that describes a task. Write a response that appropriately completes the request."

(or with the +input version of the above.)

I would like to understand why and where does it format come from (InstructGPT paper?)

Since there are already instructions in the dataset, what function does it serve to prepend this additional layer which is exactly the same for each example?

zygmuntz avatar Apr 06 '23 20:04 zygmuntz