distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

Published 20 hours ago • verified publisher icon argilla-io

Fix routing batch function deadlocks and unordered batches

Open gabrielmbmb opened this issue 9 months ago • 0 comments

Description

This PR fixes several issues that caused deadlocks and having a final dataset with unordered batches caused when a routing batch function was used in a pipeline. Several tests have been added to check that everything is working correctly.

gabrielmbmb avatar May 18 '24 11:05 gabrielmbmb