distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

[IMPROVEMENT] "Unload" steps once those are done

Open alvarobartt opened this issue 10 months ago • 0 comments

Description

Some steps, specially the Task subclasses that run a local LLM tend to consume much memory, and that should be released once the step is completed, as otherwise the accelerators use will be occupied for as long as the slower step is running.

Additionally, this implies that when a step is loaded, it's consuming memory until all the steps in the Pipeline are done, not until that step stops running / is completed.

alvarobartt avatar Apr 25 '24 09:04 alvarobartt