distilabel

distilabel copied to clipboard

Unable to use Llamacpp

3

when i run the following code on my m1 macbook pro (mac os 14.4.1 (23E224)) ```python from distilabel.pipeline import Pipeline from distilabel.llms.llamacpp import LlamaCppLLM from distilabel.steps import LoadDataFromDicts from distilabel.steps.tasks...

amritsingh183

Add `Format{Chat,Text}Generation{DPO,SFT}`

2

## Description This PR adds the following steps in order to format the batches into what the main fine-tuning frameworks / libraries (i.e. `axolotl` and `alignment-handbook`) expect for both DPO...

alvarobartt

Create `CTRL + C` signal handler for gracefully pipeline stopping

1

Create a signal handler that captures `SIGINT` (ctrl + c) and stops the pipeline gracefully.

gabrielmbmb

## Description This PR adds a new field `distilabel_meta` to store general outputs related to distilabel. Currently we will have `distilabel_id` with a UUID, and in case of `Tasks` that...

plaguss

[FEATURE] Sequential execution for local pipeline

1

## Description As mentioned by @alvarobartt and Ellamind team, it would be nice to have a sequential model for executing the pipeline, in which no multiprocessing & batching is used....

gabrielmbmb

**Is your feature request related to a problem? Please describe.** Currently, the caching system only works for full batches and doesn't seem to work across code changes. I think it...

bjoernpl

**Is your feature request related to a problem? Please describe.** I'm testing a preference pipeline with Llama3 and the parsed outputs are weird (long lists of ratings and rationales when...

dvsrepo

## Description Some steps, specially the `Task` subclasses that run a local `LLM` tend to consume much memory, and that should be released once the step is completed, as otherwise...

alvarobartt

## Description Currently `distilabel` can be only executed locally executing either a python script with the pipeline or using the CLI. For future integrations with Argilla UI or deploying `distilabel`...

gabrielmbmb

## Description Currently, the pipelines that can be executed directly from the CLI or when loaded with `Pipeline.from_yaml` are those which include only steps from `distilabel` package. The idea is...

gabrielmbmb