Agus

**Describe the bug** To simplify the code, if no name is given to a pipeline a random one is generated. **To Reproduce** Code to reproduce ```python from distilabel.pipeline import Pipeline...

[FEATURE] Update `PushToHub` to stream data to the Hub

4

**Is your feature request related to a problem? Please describe.** Some users would like to inspect the dataset as it's being created in the Hugging Face Hub. **Describe the solution...

**Is your feature request related to a problem? Please describe.** If we have a pipeline `a >> b >> c` and we create another `a >> b >> c >>...

**Is your feature request related to a problem? Please describe.** Papers like [TORA: A TOOL-INTEGRATED REASONING AGENT FOR MATHEMATICAL PROBLEM SOLVING](https://arxiv.org/pdf/2309.17452) have proven relevant to improve reasoning capabilities in math,...

**Is your feature request related to a problem? Please describe.** Take a look at [CodecLM: Aligning Language Models with Tailored Synthetic Data](https://arxiv.org/abs/2404.05875) to see if it could be integrated. **Describe...

**Is your feature request related to a problem? Please describe.** We need some integration tests for the LLMs running once a week or before a release to avoid errors with...

Add Tasks to replicate `APIGen`

2

## Description This PR adds `Tasks` to replicate [APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets](https://arxiv.org/abs/2406.18518), which yielded the following dataset: [Salesforce/xlam-function-calling-60k](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k). The following is a draft pipeline...

[EXAMPLE] Add `CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation` example

6

**Is your feature request related to a problem? Please describe.** We could create an example replicating this paper, we have most of the pieces and seems quite interesting: [CRAFT Your...

Pretty print

2

## Description This PR adds a base method to allow easily printing the prompts of the tasks. ```python from distilabel.steps.tasks import URIAL # only importable if placed at the root...

Add `CLAIR` task

2

## Description This PR adds [CLAIR](https://arxiv.org/abs/2408.06266v1) task to revise instructions: *CLAIR uses an AI system to minimally revise a solution A→A´ such that the resulting preference A `preferred` A’ is...