data_tooling icon indicating copy to clipboard operation
data_tooling copied to clipboard

Published 20 hours ago • verified publisher icon bigscience-workshop

add files to compute basic stats on pseudo crawl dataset

Open SaulLu opened this issue 2 years ago • 3 comments

As discussed with @thomasw21, this PR add basic slurm and python scripts to compute an intermiadiary metadata dataset and some statistics for the Pseudo Crawl dataset

SaulLu avatar Feb 01 '22 15:02 SaulLu