the-pile icon indicating copy to clipboard operation
the-pile copied to clipboard
verified publisher icon EleutherAI

snarky link

Open GibbilyGooo opened this issue 10 months ago • 1 comments

Is this really necessary?

This is where we send people to a bunch of gatekeepers of information. That's really cool. "Old News" yea ok

The Pile is no longer hosted or distributed by The-Eye or afils. It is however still shared among the AI/ML community presumably for its historical significance and so can still be found via public torrents.

Hashes of the original dataset can be obtained from eleuther.ai at request.

Though The Pile was removed a good while back this file was made due to regular traffic to /public/AI/pile/ from HN reposts of the original paper at eleuther.ai.

The Pile is old news, check out more recent datasets like; https://huggingface.co/datasets/bigcode/the-stack-v2

GibbilyGooo avatar Feb 27 '25 00:02 GibbilyGooo

Does anyone have a torrent or magnet link handy? Finding this was a kick to the face :(

m0xsec avatar Mar 03 '25 00:03 m0xsec