tfrecord-viewer icon indicating copy to clipboard operation
tfrecord-viewer copied to clipboard

Not working on imagenet image

Open weiyshay opened this issue 2 years ago • 2 comments

I ran to error with the followinig error:

python3 ./tfviewer.py /datasets/imagenet2012/5.1.0/imagenet2012-train.tfrecord-00004 --overlay classification

imagenet2012-train.tfrecord-00000-of-01024 imagenet2012-train.tfrecord-00003-of-01024 imagenet2012-train.tfrecord-00006-of-01024 imagenet2012-train.tfrecord-00009-of-01024 imagenet2012-train.tfrecord-00001-of-01024 imagenet2012-train.tfrecord-00004-of-01024 imagenet2012-train.tfrecord-00007-of-01024
imagenet2012-train.tfrecord-00002-of-01024 imagenet2012-train.tfrecord-00005-of-01024 imagenet2012-train.tfrecord-00008-of-01024
root@uefi-popos-20:/tfrecord-viewer/src/tfrecord-viewer# python3 ./tfviewer.py /datasets/imagenet2012/5.1.0/imagenet2012-train.tfrecord-00000-of-01024 --overlay classification Pre-loading up to 200 examples.. 2022-04-14 12:59:53.259380: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.264222: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.264642: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.265321: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-04-14 12:59:53.265464: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.265895: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.266267: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.719396: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.719827: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.720189: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-04-14 12:59:53.720539: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 6664 MB memory: -> device: 0, name: NVIDIA GeForce RTX 2070 SUPER, pci bus id: 0000:06:00.0, compute capability: 7.5 Filename: /datasets/imagenet2012/5.1.0/imagenet2012-train.tfrecord-00000-of-01024 Traceback (most recent call last): File "./tfviewer.py", line 192, in count = preload_images(args.max_images) File "./tfviewer.py", line 116, in preload_images image_bytes = example.features.feature[args.image_key].bytes_list.value[0] IndexError: list index (0) out of range

weiyshay avatar Apr 14 '22 13:04 weiyshay

Getting the same error, any resolution yet?

MannyBoink avatar Oct 04 '22 20:10 MannyBoink

your default names are no the same , for this 0 len list by default the keys names are "image" and _"file_name" but the tool have :

parser.add_argument('--image-key', type=str, default="image/encoded",
                    help='Key to the encoded image.')
parser.add_argument('--filename-key', type=str, default="image/filename",
                    help='Key to the unique ID of each record.')

IMPORTANT the tfrecord have key words inside by default , you should respect it

tf_example = tf.train.Example(features=tf.train.Features(feature={
    'image/height': dataset_util.int64_feature(height),
    'image/width': dataset_util.int64_feature(width),
    'image/filename': dataset_util.bytes_feature(filename),
    'image/source_id': dataset_util.bytes_feature(filename),
    'image/encoded': dataset_util.bytes_feature(encoded_jpg),
    'image/format': dataset_util.bytes_feature(image_format),
    'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
    'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
    'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
    'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
    'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
    'image/object/class/label': dataset_util.int64_list_feature(classes),
})) 

better use default script (COCO to tfrecord for exmaple https://github.com/tensorflow/models/blob/master/official/vision/data/create_coco_tf_record.py ) or object detection API scripts to create a tfrecord https://github.com/tensorflow/models/tree/master/research/object_detection/dataset_tools

Leci37 avatar Jan 02 '24 18:01 Leci37