how to transfer imagenet raw datasets into corresponding rfrecords?

[HamLaertes]

trafficstars

Hi, when I ran this programe, it said 'cannot found imagenet2012-train.tfrecord-00823-of-01000' error, how can I transfer imagenet_train.tar and imagenet_val.tar into corresponding tfrecord?

[lgeiger]

We use TensorFlow Datasets for preparing the ImageNet dataset. If I recall correctly they used to be downloaded automatically. Recently Tensorflow Datasets switched to allow to use already downloaded tar files. You can find documentation for how to do this here.

[HamLaertes]

We use TensorFlow Datasets for preparing the ImageNet dataset. If I recall correctly they used to be downloaded automatically. Recently Tensorflow Datasets switched to allow to use already downloaded tar files. You can find documentation for how to do this here.

thanks, but tfds seems to load imagenet as tfrecord. So how can I transfer raw imagenet tars to those tfrecords?

[HamLaertes]

We use TensorFlow Datasets for preparing the ImageNet dataset. If I recall correctly they used to be downloaded automatically. Recently Tensorflow Datasets switched to allow to use already downloaded tar files. You can find documentation for how to do this here.

Hi, thanks for replying. I have already ran this code on my server, it seems to work fine. However, after checking on tensorflow_datasets github, I found there may be a bug in your codes requirments. You need tensorflow-datasets:1.2.0, but under 1.2.0, it cannot successfully preprocess imagenet and get corresponding tfrecord. This bug was fixed under version 1.3.1, but still cannot work (with some other problems). I recommend you to update the version of tensorflow-datasets to 3.0.0, and the version of imagenet thus no longer 2.0.1 but 5.0.0. After doing such aprovements, this programe ran on my server succesfully.

My server: Centos. Python version: 3.6.9. Tensorflow-gpu:2.0.0.