unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Fix Llama3 format

Open elronbandel opened this issue 1 year ago • 1 comments

elronbandel avatar Jun 24 '24 20:06 elronbandel

I removed the formats.llama3_chat_with_system_prompt system prompt that was not being used anywhere, and also wasn't right (e.g. had \n between parts instead of \n\n).

I've updated formats.llama3_chat definition so:

  1. the demos also have the user/assistant prompts. I believe that is how we are supposed to provide examples.
  2. added a default DEFAULT_SYSTEM_PROMPT that is one that is being suggested/used for llama2, e.g. see: https://developer.ibm.com/tutorials/awb-prompt-engineering-llama-2/

I've added a formats.llama3_instruct that I believe more accurately reflects the official guidance https://huggingface.co/blog/llama3#how-to-prompt-llama-3 which states

The Instruct versions use the following conversation structure:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{{ system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ user_msg_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ model_answer_1 }}<|eot_id|>

This format has to be exactly reproduced for effective use.

Note: the instruction is not a part of the demos. I believe a demo example-specific instruction can improve the prompt, but I don't think it's possible the way unitxt templates work now (instruction cannot be used in demo_format).

oktie avatar Jun 25 '24 14:06 oktie