distilabel
distilabel copied to clipboard
[FEATURE] improved `Task` `prompt` customization
Is your feature request related to a problem? Please describe.
- where can I find the system prompt?
- how easy is it to overwrite/adapt the prompt template?
- can I leave out certain parts of the generation e.g. rationale/critique.
Describe the solution you'd like Some features/improvements that help with the questions above.
Describe alternatives you've considered NA
Additional context
ideas which might not need to work with set_runtime_parameters
specifically.
- would it be possible to have a
few-shot
example section
Task.set_runtime_parameters(
"few_shot_example": {
"inputs": {"input_column": ["my example input"]}},
"outputs": {"outut_column": ["my example output"]}}
}
)
# or
Task.set_runtime_parameters(
"few_shot_example": "my good example"
)
- can we allow adding a pre-fix/suf-fix to each prompt by default.
Task.set_runtime_parameters(
"prompt_prefix": "always do this thingy",
"prompt_suffix": "always do that other thingy"
)
- always allow for adding a system prompt, even when this was not added in the original paper
Task.set_runtime_parameters(
"system_prompt": "always do this thingy"
)
- Overwrite prompt templates. should wee allow disabling certain output columns present in the prompt template through if-else statements? E.g., in UltraFeedback I might not want rationales.
Task.set_runtime_parameters(
"prompt template": "do ultra feedback {score} raationale {rationale}"
)
Task.set_runtime_parameters(
"prompt template": "do ultra feedback {score}"
)
Task.set_runtime_parameters(
"prompt template": "path/to/.jinja2"
)