IMDb-Face Downloading is too slow.

Great job! I use python urllib. Maybe I am in China, the url for downloading is too slow. Is there any way to deal with it? or is there anyone to share the dataset? Thanks.

Sep 20 '18 13:09 LCorleone

I think you can use proxy servers to accelerate your access to these images on Amazon.

Sep 25 '18 09:09 hwfan

@LCorleone You can use multi-threading or proxy servers in python to speed up downloading images. The code is available.

Sep 30 '18 08:09 braveapple

@BraveApple Thanks，nice work！

Oct 06 '18 07:10 LCorleone

i dont know why i must use proxy servers to download picture, it is maddening when requests post break

Oct 24 '18 03:10 smartwell

i dont know why i must use proxy servers to download picture, it is maddening when requests post break

i give up

Oct 24 '18 03:10 smartwell

Actually, the best way to download such datasets is to use cloud server. I used to use AWS to do this. However, there is still a problem waiting for us. It's very slow to upload datasets to our computer in China. Even using bypy to do this, it still sucks!

Nov 08 '18 06:11 wangx404

I wrote a script yesterday to download the dataset on AWS. After 12 hours, 600k images have been downloaded. (About 20% of the image links no long exist.) Even I have croped the face from the raw image, the dataset is still very huge. I think it would have a size of 55G when the whole dataset was downloaded.

Nov 09 '18 01:11 wangx404

Finally, I finished. It's about 50G, with about 17% links expired.

Nov 11 '18 05:11 wangx404

@wangx404 could you share the cropped data?