batched data

[aldopareja]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit report (我已阅读并同意 Language Policy).
• [X] Pleas do not modify this template :) and fill in all the required fields.

Provide a description of requested docs changes

Sometimes you generate workflows that should be applied to many different datapoints or a whole dataset simil to dataset.map(apply_workflow) that you do with hf Dataset. You load a datapoint and you want to apply the same workflow to each datapoint (specifying a column like prompt). I don't think that is doable at the moment and it's a very basic use case of a workflow for data manipulation.

[takatost]

What's your use case and what do you need to do? 🤔

[galer7]

Not OP, but one of my use cases is:

• text input -> LLM to split into multiple strings e.g. ToC as input and I get an array with the chapters/subchapters
• I want to apply a workflow to each generated section individually

[lazyFrogLOL]

This feature works like map reduce. After data input into the workflow, multiple LLMs generate multiple results, which are then aggregated through reduce to form a final result.

[crazywoola]

See this also, https://github.com/langgenius/dify/issues/4275, will close this one.