SPM_toolkit icon indicating copy to clipboard operation
SPM_toolkit copied to clipboard

Published 20 hours ago • verified publisher icon lanwuwei

load_word_vectors problems

Open funnytestingcase opened this issue 4 years ago • 4 comments

I have torchtext 0.1.1 , python 2.7 but function "load_word_vectors" would not download the correct zip file (the 2.18GB one) when it starts downloading "glove.840B.300d: 8.19kB [00:00, 9.51kB/s]" and it always ends up with a bad zip file

was I missing anything?

funnytestingcase avatar Dec 25 '20 09:12 funnytestingcase

torchtext 0.1.1 is correct, how about your network speed?

lanwuwei avatar Dec 26 '20 16:12 lanwuwei

I solved this error by modifying vocab.py file from torchtext, basically I bypassed the downloading the unzip step.

and, What's your memory size when implementing those four frameworks? My 4GB was quickly run out.

funnytestingcase avatar Dec 28 '20 06:12 funnytestingcase

4GB is too small, I ran this code in a server with tens of GB of memory. ESIM is memory efficient, as it uses the file iterator without loading all the data into memory. You can try ESIM first with your 4GB memory.

lanwuwei avatar Dec 28 '20 06:12 lanwuwei

thx for the quick reply !

funnytestingcase avatar Dec 28 '20 06:12 funnytestingcase