AutoPrompt icon indicating copy to clipboard operation
AutoPrompt copied to clipboard

how to set the eval part in generation tasks

Open tfk12 opened this issue 7 months ago • 14 comments

In face of generation situations, I guess we should set function name to ranking. And as so we should set function_params. That's not so obvious.

the following not works for me.

Can we have a concrete demo for generation tasks? Thanks in advance

eval: function_name: 'ranking' function_params: prompt: 'prompts/predictor_completion/prediction_generation.prompt' mini_batch_size: 1 llm: type: '...' num_workers: 10 mode: 'verbose' label_schema: ["1","2","3","4","5"]

tfk12 avatar Jul 16 '24 09:07 tfk12