data_tooling icon indicating copy to clipboard operation
data_tooling copied to clipboard

Published 20 hours ago • verified publisher icon bigscience-workshop

add files to compute basic stats on pseudo crawl dataset

Open SaulLu opened this issue 3 years ago • 3 comments

As discussed with @thomasw21, this PR add basic slurm and python scripts to compute an intermiadiary metadata dataset and some statistics for the Pseudo Crawl dataset

SaulLu avatar Feb 01 '22 15:02 SaulLu

@SaulLu can you edit and merge? Thanks!

HugoLaurencon avatar Feb 08 '22 12:02 HugoLaurencon

@HugoLaurencon, do you need it urgently ?

SaulLu avatar Feb 08 '22 19:02 SaulLu

@SaulLu No, no worries! It was just to tag you in case you didn't see the comments from Thomas

HugoLaurencon avatar Feb 09 '22 10:02 HugoLaurencon