neural-storyteller
neural-storyteller copied to clipboard
ryankiros
BookCorpus dataset gone
The BookCorpus dataset located at http://www.cs.toronto.edu/~mbweb/ returns a 403. Can you provide a mirror or at the very least give a few lines as an example of how the dataset needs to be formatted?
What about using the google big query? Its freaking huge.
On Dec 6, 2016 2:51 AM, "Nao Tokui" [email protected] wrote:
this one? http://yknzhu.wixsite.com/mbweb
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ryankiros/neural-storyteller/issues/17#issuecomment-265106922, or mute the thread https://github.com/notifications/unsubscribe-auth/AVOF0KDf0NhIQkje0mAk63vyZoYWo2Jsks5rFTAngaJpZM4KzEZh .
I went to the website and it said we'd have to collect it ourselves if we want the dataset. I wonder if anyone knows what happened?
I also wonder if I must collect the dataset myself. Is there any other source to get the dataset?
The sad thing is there are new papers that reference this dataset and reference the old paper for the corpus. The dataset has vanished from their website and there are no answers from any of the authors. I am searching and it is not available anywhere.
@TZWwww I wrote a crawler for the original site, smashwords.com. Please use this. https://github.com/soskek/bookcorpus
Do you think it is possible to host the dataset for a while? I run into HTTP errors on some of the books, so this script ends up downloading only 7000/11000 books.
I cannot. Instead, see another option: https://github.com/soskek/bookcorpus/issues/24#issuecomment-556024973
Would be ideal, if it were via distributed P2P file sharing systems.
