Steering/control vectors not working with Llama 3 chat models

[rahulb99]

Model: meta-llama/Meta-Llama-3-8B Code: examples/primary_emotions/emotion_concept.ipynb and others

coefficient: 1.0 to 3.0

[Winston-Yuan]

Have you solved this problem?

[HenryCai11]

Hi @Winston-Yuan @rahulb99 . Just for your information, you need to use the correct template for Meta-Llama-3.1-8B-Instruct:

template = "<|start_header_id|>user<|end_header_id|>\n\n {instruction} <eot_id><|start_header_id|>assistant<|end_header_id|>\n\n"

And in some cases, setting tokenizer.padding_side to right may cause the problem of generating "!!!!!!" which are actually eos_tokens