distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

Published 20 hours ago • verified publisher icon argilla-io

[FEATURE] Return generation statistics

Open gabrielmbmb opened this issue 1 year ago • 1 comments

Along with the generated dataset, it would be good to return a data structure containing statistics of the generation such as elapsed time, total tokens generated by the labeller, etc.

gabrielmbmb avatar Nov 14 '23 16:11 gabrielmbmb

IMO we can add those within the DatasetCard to be generated too, as well as in a separate file as statistics.json or similar.

alvarobartt avatar Nov 20 '23 09:11 alvarobartt