distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

[FEATURE] Enable debugging cols similar to the previous raw_generation and labelling_prompt

Open dvsrepo opened this issue 10 months ago • 0 comments

Is your feature request related to a problem? Please describe.

I'm testing a preference pipeline with Llama3 and the parsed outputs are weird (long lists of ratings and rationales when my number of generations is two). I'd like to have a way to automatically enable some columns in my resulting distiset to inspect:

  1. The prompt sent by the UltraFeedback task
  2. The raw responses parsed with the UF task

This is of course applicable to other Steps but it's more crucial when using tasks with complex prompt templates and output parsing.

Describe the solution you'd like Something like enable debug columns when I setup the Task

Describe alternatives you've considered Please suggest alternatives I can use in the meantime

Additional context Add any other context or screenshots about the feature request here.

dvsrepo avatar Apr 25 '24 10:04 dvsrepo