opencompass
opencompass copied to clipboard
请问我想对cmmlu这类选择题的模板进行更改,但是system变量无法赋值
Prerequisite
- [X] I have searched Issues and Discussions but cannot get the expected help.
- [X] The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
cmmlu_infer_cfg = dict(
ice_template=dict(
type=PromptTemplate,
template=dict(
begin=[dict(role='SYSTEM', fallback_role='HUMAN', prompt='以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项。'),'</E>'],
#begin="</E>",
round=[
dict(
role="HUMAN",
prompt=
f"USER: {{question}}\nA. {{A}}\nB. {{B}}\nC. {{C}}\nD. {{D}}"
),
dict(role="BOT", prompt=' ASSISTANT: 答案是: {answer}</s>'),
]),
ice_token="</E>",
),
)
Reproduces the problem - code/configuration sample
请问怎么让begin里的_ch_name进行赋值呢,我看只有round里面会赋值。或者我应该怎么写比较好呢。”以下是关于{_ch_name}的单项选择题,请直接给出正确答案的选项“在fewshot中只在最前面出现一次
Reproduces the problem - command or script
1
Reproduces the problem - error message
1
Other information
1
我希望的模板是 以下是关于{法律}的单项选择题,请直接给出正确答案的选项。 {qustion1} {A} {B} {C} {D} 答案是:{answer} {qustion2} {A} {B} {C} {D} 答案是:{answer} ''' {question} {A} {B} {C} {D} 答案是: