the-pile icon indicating copy to clipboard operation
the-pile copied to clipboard

Published 20 hours ago • verified publisher icon EleutherAI

"Github" code data download only

Open HangXue-lab opened this issue 2 years ago • 2 comments

The size of pile is too big for me. I just want to download the "Github" code data. But the number of Pile train file is 30. I would like to know exactly which file contains the "Github" code data.

HangXue-lab avatar Jan 17 '23 14:01 HangXue-lab

The data is already processed by that stage, and may not be what you want. You probably want the github.tar from the preliminary components https://the-eye.eu/public/AI/pile_preliminary_components/github.tar and process it yourself.

igorbrigadir avatar Jan 17 '23 16:01 igorbrigadir

The link is no longer working, is there another link to obtain the data?

osainz59 avatar Oct 30 '23 08:10 osainz59