yfcc100m-entity icon indicating copy to clipboard operation
yfcc100m-entity copied to clipboard

Can you share the source file data of YFCC100M, that is, the public urls?

Open Chen-Song opened this issue 5 years ago • 3 comments

Hi, Can you share the source file data of YFCC100M, that is, the public urls? It may be because of the network of mine, the metadata of the YFCC100M is often wrong. So I hope you share the metadata with me. Thanks.

Chen-Song avatar Sep 07 '19 10:09 Chen-Song

Can you post a link that does not work for you?

raingo avatar Sep 07 '19 23:09 raingo

Hi I downloaded the dataset as follows:

Run "pip install awscli" to install awscli. Run "aws configure" and enter access key and secret (available via https://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key). Run "aws s3 ls s3://yahoo-webscope-i3/" to view the S3 objects for I3 - Yahoo Flickr Creative Commons 100M (14G) (Hosted on AWS). Run "aws s3 cp s3://yahoo-webscope-i3 . --recursive" to download I3 - Yahoo Flickr Creative Commons 100M (14G) (Hosted on AWS) to current directory.

But there were a lot of failures. First, my aws version is 'aws-cli/1.16.234 Python/3.6.5 Linux/4.15.0-55-generic botocore/1.12.224' and in 'aws configure' command, I set the 'region name=us-east-2'. In fact, my location is Beijing, China, but when I set the region name= cn-north-1, I get an error message that the access ID is not available. So I have to set the region to us-east-2.

Second, I set the 'Default output format=json'. Third, I run aws s3 cp s3://yahoo-webscope-i3 . --recursive and the download terminal as follows:

download: s3://yahoo-webscope-i3/WebscopeReadMe.txt to ./WebscopeReadMe.txt download failed: s3://yahoo-webscope-i3/yfcc100m_places.bz2 to ./yfcc100m_places.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_autotags.bz2 to ./yfcc100m_autotags.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_exif.bz2 to ./yfcc100m_exif.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_dataset.bz2 to ./yfcc100m_dataset.bz2 Connect timeout on endpoint URL: "https://yahoo-webscope-i3.s3.amazonaws.com/yfcc100m_dataset.bz2"

Look forward your reply. Thanks. Best, SongChen.

Chen-Song avatar Sep 08 '19 07:09 Chen-Song

Hi I downloaded the dataset as follows:

Run "pip install awscli" to install awscli. Run "aws configure" and enter access key and secret (available via https://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key). Run "aws s3 ls s3://yahoo-webscope-i3/" to view the S3 objects for I3 - Yahoo Flickr Creative Commons 100M (14G) (Hosted on AWS). Run "aws s3 cp s3://yahoo-webscope-i3 . --recursive" to download I3 - Yahoo Flickr Creative Commons 100M (14G) (Hosted on AWS) to current directory.

But there were a lot of failures. First, my aws version is 'aws-cli/1.16.234 Python/3.6.5 Linux/4.15.0-55-generic botocore/1.12.224' and in 'aws configure' command, I set the 'region name=us-east-2'. In fact, my location is Beijing, China, but when I set the region name= cn-north-1, I get an error message that the access ID is not available. So I have to set the region to us-east-2.

Second, I set the 'Default output format=json'. Third, I run aws s3 cp s3://yahoo-webscope-i3 . --recursive and the download terminal as follows:

download: s3://yahoo-webscope-i3/WebscopeReadMe.txt to ./WebscopeReadMe.txt download failed: s3://yahoo-webscope-i3/yfcc100m_places.bz2 to ./yfcc100m_places.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_autotags.bz2 to ./yfcc100m_autotags.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_exif.bz2 to ./yfcc100m_exif.bz2 Max Retries Exceeded download failed: s3://yahoo-webscope-i3/yfcc100m_dataset.bz2 to ./yfcc100m_dataset.bz2 Connect timeout on endpoint URL: "https://yahoo-webscope-i3.s3.amazonaws.com/yfcc100m_dataset.bz2"

Look forward your reply. Thanks. Best, SongChen.

have you downloaded these metadata? The data cannot be downloaded on the official website. if you have made it, look forward for your share.

BeCarefulOfYournaoke avatar May 17 '24 14:05 BeCarefulOfYournaoke