tf_object_detection_cm
tf_object_detection_cm copied to clipboard
svpino
ValueError: Python inputs incompatible with input_signature:
ValueError: Python inputs incompatible with input_signature: inputs: ( [[[ 39 38 37 ... 36 36 36] [ 38 38 37 ... 36 36 36] [ 38 38 37 ... 36 36 36] ... [ 23 22 22 ... 163 158 155] [ 23 22 22 ... 157 153 151] [ 23 22 22 ... 155 153 152]]]) input_signature: ( TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor'))
The error appears because the input_tensor shape isn't being loaded correctly. You need to convert your image being loaded into the correct shape (1, None, None, 3). I made the following changes to fix it:
This begins from line 122 of confusion_matrix_tf2.py
image = decoded_dict[fields.InputDataFields.image]
img = Image.open(io.BytesIO(image)).convert("RGB")
input_tensor = tf.convert_to_tensor(img)
input_tensor = input_tensor[tf.newaxis, ...]
Note that, after this amendment, you will have to make another change in the draw function because the input_tensor no longer behaves as a numpy array. To solve this, simply add the following above line 239:
image = np.asarray(image)
Hope this helps. Took me a while to figure out the root of the problem too.
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed
https://github.com/zezo357/tf_object_detection_cm
image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]
I have done the way you have said. But I still face this error while generating the confusion matrix.. please do help me out too.
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed
https://github.com/zezo357/tf_object_detection_cm
how did you solve your problem please do help me too please. I am facing the same error
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed
https://github.com/zezo357/tf_object_detection_cm
how did you solve your problem please do help me too please. I am facing the same error
I did follow the same code provided in the comment before me
And you can check the fork (I think I edited something more to make it work)
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed
https://github.com/zezo357/tf_object_detection_cm
how did you solve your problem please do help me too please. I am facing the same error
Checked again I did the same as the comment
Did you change the code at the line two hundred? The np.array one ?
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed https://github.com/zezo357/tf_object_detection_cm
how did you solve your problem please do help me too please. I am facing the same error
Checked again I did the same as the comment
Did you change the code at the line two hundred? The np.array one ?
Here is my complete code of confusion_matrix_tf2.csv import itertools import os from PIL import Image import io import cv2 import tensorflow as tf import numpy as np import pandas as pd import time, progressbar from object_detection.inference import detection_inference from object_detection.utils.dataset_util import bytes_list_feature from object_detection.utils.dataset_util import float_list_feature from object_detection.utils.dataset_util import int64_list_feature from object_detection.utils.dataset_util import int64_feature from object_detection.utils.dataset_util import bytes_feature from object_detection.utils import label_map_util from object_detection.utils import visualization_utils as viz_utils from object_detection.core import data_parser from object_detection.core import standard_fields as fields
from object_detection.metrics.tf_example_parser import BoundingBoxParser, StringParser, Int64Parser, FloatParser
tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images')
FLAGS = tf.compat.v1.flags.FLAGS IOU_THRESHOLD = 0.5 CONFIDENCE_THRESHOLD = 0.5
class CustomParser(data_parser.DataToNumpyParser): """Tensorflow Example proto parser."""
def init(self): self.items_to_handlers = { fields.InputDataFields.image: StringParser( fields.TfExampleFields.image_encoded), fields.InputDataFields.groundtruth_boxes: BoundingBoxParser( fields.TfExampleFields.object_bbox_xmin, fields.TfExampleFields.object_bbox_ymin, fields.TfExampleFields.object_bbox_xmax, fields.TfExampleFields.object_bbox_ymax), fields.InputDataFields.groundtruth_classes: Int64Parser( fields.TfExampleFields.object_class_label) } # Optional self.filename = StringParser(fields.TfExampleFields.filename)
def parse(self, tf_example): """ Parses tensorflow example and returns a tensor dictionary. Args: tf_example: a tf.Example object. Returns: A dictionary of the following numpy arrays: image - string containing input image. filename - string containing input filename (optional, None if not specified) groundtruth_boxes - a numpy array containing groundtruth boxes. groundtruth_classes - a numpy array containing groundtruth classes. """ results_dict = {} parsed = True for key, parser in self.items_to_handlers.items(): results_dict[key] = parser.parse(tf_example) parsed &= (results_dict[key] is not None)
# TODO: need to test
filename = self.filename.parse(tf_example)
results_dict['filename'] = filename # could be None
return results_dict if parsed else None
def compute_iou(groundtruth_box, detection_box): g_ymin, g_xmin, g_ymax, g_xmax = tuple(groundtruth_box.tolist()) d_ymin, d_xmin, d_ymax, d_xmax = tuple(detection_box.tolist())
xa = max(g_xmin, d_xmin)
ya = max(g_ymin, d_ymin)
xb = min(g_xmax, d_xmax)
yb = min(g_ymax, d_ymax)
intersection = max(0, xb - xa + 1) * max(0, yb - ya + 1)
boxAArea = (g_xmax - g_xmin + 1) * (g_ymax - g_ymin + 1)
boxBArea = (d_xmax - d_xmin + 1) * (d_ymax - d_ymin + 1)
return intersection / float(boxAArea + boxBArea - intersection)
def process_detections(input_dataset, model, categories, draw_option, draw_save_path):
def process_detections(input_tfrecord_path, model, categories, draw_option, draw_save_path): """ Creates input dataset from tfrecord, runs detection model, compares detection results with ground truth Args: input_tfrecord_path: path of input tfrecord file model: path of detection model .pb file categories: ordered array of class IDs draw_option: whether or not to visualize and save detections and ground truth boxes draw_save_path: where to save visualizations if draw_option is true """ data_parser = CustomParser() confusion_matrix = np.zeros(shape=(len(categories) + 1, len(categories) + 1))
# Create dataset from records
input_dataset = tf.data.TFRecordDataset(input_tfrecord_path)
with progressbar.ProgressBar(max_value=progressbar.UnknownLength) as bar:
for image_index, record in enumerate(input_dataset):
example = tf.train.Example()
example.ParseFromString(record.numpy())
decoded_dict = data_parser.parse(example)
if decoded_dict:
image = decoded_dict[fields.InputDataFields.image]
input_tensor = np.expand_dims(Image.open(io.BytesIO(image)), axis=0)
img = Image.open(io.BytesIO(image)).convert("RGB")
input_tensor = tf.convert_to_tensor(img)
input_tensor = input_tensor[tf.newaxis, ...]
groundtruth_boxes = decoded_dict[fields.InputDataFields.groundtruth_boxes]
groundtruth_classes = decoded_dict[fields.InputDataFields.groundtruth_classes].astype('uint8')
detections = model(input_tensor) # Run model inference
detection_scores = detections['detection_scores'][0].numpy()
detection_boxes = detections['detection_boxes'][0].numpy()[detection_scores >= CONFIDENCE_THRESHOLD]
detection_classes = detections['detection_classes'][0].numpy()[detection_scores >= CONFIDENCE_THRESHOLD].astype('uint8')
filename = decoded_dict[fields.InputDataFields.filename]
filename = filename.decode('UTF-8') if filename is not None else f'image-{image_index}.png'
matches = []
image_index += 1
for i, groundtruth_box in enumerate(groundtruth_boxes):
for j, detection_box in enumerate(detection_boxes):
iou = compute_iou(groundtruth_box, detection_box)
if iou > IOU_THRESHOLD:
matches.append([i, j, iou])
matches = np.array(matches)
if matches.shape[0] > 0:
# Sort list of matches by descending IOU so we can remove duplicate detections
# while keeping the highest IOU entry.
matches = matches[matches[:, 2].argsort()[::-1][:len(matches)]]
# Remove duplicate detections from the list.
matches = matches[np.unique(matches[:,1], return_index=True)[1]]
# Sort the list again by descending IOU. Removing duplicates doesn't preserve
# our previous sort.
matches = matches[matches[:, 2].argsort()[::-1][:len(matches)]]
# Remove duplicate ground truths from the list.
matches = matches[np.unique(matches[:,0], return_index=True)[1]]
for i in range(len(groundtruth_boxes)):
if matches.shape[0] > 0 and matches[matches[:,0] == i].shape[0] == 1:
confusion_matrix[groundtruth_classes[i] - 1][int(detection_classes[int(matches[matches[:,0] == i, 1][0])] - 1)] += 1
else:
confusion_matrix[groundtruth_classes[i] - 1][confusion_matrix.shape[1] - 1] += 1
for i in range(len(detection_boxes)):
if matches.shape[0] > 0 and matches[matches[:,1] == i].shape[0] == 0:
confusion_matrix[confusion_matrix.shape[0] - 1][int(detection_classes[i] - 1)] += 1
if draw_option:
draw(filename, draw_save_path, input_tensor[0], categories,
groundtruth_boxes, groundtruth_classes, detection_boxes, detection_classes, detection_scores)
else:
print(f'Skipping image {image_index}')
bar.update(image_index)
print(f'Processed {image_index + 1} images')
return confusion_matrix
def display(confusion_matrix, categories, output_path): ''' Displays confusion matrix as pandas df to terminal and saves as CSV Args: confusion_matrix: matrix to be displayed categories: ordered array of class IDs output_path: where to save CSV ''' print('\nConfusion Matrix:') print(confusion_matrix, '\n') results = []
for i in range(len(categories)):
id = categories[i]['id'] - 1
name = categories[i]['name']
total_target = np.sum(confusion_matrix[id,:])
total_predicted = np.sum(confusion_matrix[:,id])
precision = float(confusion_matrix[id, id] / total_predicted)
recall = float(confusion_matrix[id, id] / total_target)
results.append({'category' : name, f'precision_@{IOU_THRESHOLD}IOU' : precision, f'recall_@{IOU_THRESHOLD}IOU' : recall})
df = pd.DataFrame(results)
print(df)
df.to_csv(output_path)
def draw(image_name, image_path, image, categories, groundtruth_boxes, groundtruth_classes, detection_boxes, detection_classes, detection_scores): ''' Draws ground truth and detection boxes and labels onto image, saves image Args: image_name: what to save the image as image_path: where to save the image image: image as np array categories: ordered array of class IDs for detections groundtruth_boxes: coordinates of ground truth boxes groundtruth_classes: class IDs corresponding to "categories" detection_boxes: coordinates of detection boxes from model detection_classes: class IDs correpsonding to "categories" detection_scores: scores for each detection box ''' # Convert category array to category index cat_index = {} for i, item in enumerate(categories): cat_index[i+1] = item
# Draw ground truth
'''
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
NOTE: as of 11.17.2020, viz_util's ground truth
option won't print the class name with the box.
To see the class name for ground truth boxes, add
class_name = category_index[classes[i]]['name']
box_to_display_str_map[box].append(class_name)
under "if scores is None:" (line 1182)
'''
image = np.asarray(image)
image_viz = image.copy()
viz_utils.visualize_boxes_and_labels_on_image_array(
image_viz,
groundtruth_boxes,
groundtruth_classes,
None, # scores = None triggers ground truth mode
cat_index,
use_normalized_coordinates=True,
max_boxes_to_draw=200,
min_score_thresh=CONFIDENCE_THRESHOLD,
agnostic_mode=False,
line_thickness=3,
groundtruth_box_visualization_color='blue') # opencv will make this red
# Draw detections
viz_utils.visualize_boxes_and_labels_on_image_array(
image_viz,
detection_boxes,
detection_classes,
detection_scores,
cat_index,
use_normalized_coordinates=True,
max_boxes_to_draw=200,
min_score_thresh=CONFIDENCE_THRESHOLD,
agnostic_mode=False,
line_thickness=1)
cv2.imwrite(os.path.join(image_path, image_name), image_viz)
def main(_): required_flags = ['input_tfrecord_path', 'output_path', 'inference_graph', 'class_labels'] for flag_name in required_flags: if not getattr(FLAGS, flag_name): raise ValueError(f'Flag --{flag_name} is required')
input_tfrecord_path = FLAGS.input_tfrecord_path
print(input_tfrecord_path)
draw_option = FLAGS.draw_option.lower() == 'true'
draw_save_path = FLAGS.draw_save_path
if draw_option:
if draw_save_path is None:
raise ValueError('If --draw_option is True, --draw_save_path is required')
# Get class names
label_map = label_map_util.load_labelmap(FLAGS.class_labels)
categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=100, use_display_name=True)
# Load model
print('Loading model...')
model = tf.saved_model.load(FLAGS.inference_graph)
# Run inference and compute confusion matrix
print('Evaluating model...')
confusion_matrix = process_detections(input_tfrecord_path, model, categories, draw_option, draw_save_path)
# Save to CSV
print('Saving confusion matrix...')
display(confusion_matrix, categories, FLAGS.output_path)
print('Done!')
if name == 'main': tf.compat.v1.app.run()
And here is my error
Traceback (most recent call last):
File "confusion_matrix_tf2.py", line 306, in
it should be
if decoded_dict:
image = decoded_dict[fields.InputDataFields.image]
img = Image.open(io.BytesIO(image)).convert("RGB")
img = np.asarray(img)
input_tensor = tf.convert_to_tensor(img)
input_tensor = input_tensor[tf.newaxis, ...]
example
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Thank you very much i created a fork with your fix and its now working ♥ there was some numpy error but i added a line and it got fixed
https://github.com/zezo357/tf_object_detection_cm
Oh man thank you sooooooooooooooo much. I just copied your all edited code and boom my confusion matrix was shown.. Thanks alot.
Can you please just have a look on my confusion matrix that it is correct or not?? It has two classes only
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Can you explain that why my confusion matrix has shown 3 rows 3 columns. Although I have 2 classes. Evaluating model... | | # | 624 Elapsed Time: 0:00:28 Processed 625 images Saving confusion matrix...
Confusion Matrix: [[199. 35. 0.] [ 43. 347. 0.] [ 45. 76. 0.]]
category ... [email protected]
0 NORMAL ... 0.850427 1 PNEUMONIA ... 0.889744
[2 rows x 3 columns] Done!
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Can you explain that why my confusion matrix has shown 3 rows 3 columns. Although I have 2 classes. Evaluating model... | | # | 624 Elapsed Time: 0:00:28 Processed 625 images Saving confusion matrix...
Confusion Matrix: [[199. 35. 0.] [ 43. 347. 0.] [ 45. 76. 0.]]
category ... [email protected]0 NORMAL ... 0.850427 1 PNEUMONIA ... 0.889744
[2 rows x 3 columns] Done!
It should show 2 row 2 col
I don't know why the third one is there
U can check your inputs it may have a third label or something (I don't really know 🥲)
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Can you explain that why my confusion matrix has shown 3 rows 3 columns. Although I have 2 classes. Evaluating model... | | # | 624 Elapsed Time: 0:00:28 Processed 625 images Saving confusion matrix... Confusion Matrix: [[199. 35. 0.] [ 43. 347. 0.] [ 45. 76. 0.]]
category ... [email protected]0 NORMAL ... 0.850427 1 PNEUMONIA ... 0.889744 [2 rows x 3 columns] Done!
It should show 2 row 2 col
I don't know why the third one is there
U can check your inputs it may have a third label or something (I don't really know 🥲)
It is explain in one of the issues of this same script.
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Can you explain that why my confusion matrix has shown 3 rows 3 columns. Although I have 2 classes. Evaluating model... | | # | 624 Elapsed Time: 0:00:28 Processed 625 images Saving confusion matrix... Confusion Matrix: [[199. 35. 0.] [ 43. 347. 0.] [ 45. 76. 0.]]
category ... [email protected]0 NORMAL ... 0.850427 1 PNEUMONIA ... 0.889744 [2 rows x 3 columns] Done!
It should show 2 row 2 col
I don't know why the third one is there
U can check your inputs it may have a third label or something (I don't really know 🥲)
Can you please let me know how it has calculated precision and recall for separate class, like separate precision for normal class and separate precision for pneumonia class??
it should be
if decoded_dict: image = decoded_dict[fields.InputDataFields.image] img = Image.open(io.BytesIO(image)).convert("RGB") img = np.asarray(img) input_tensor = tf.convert_to_tensor(img) input_tensor = input_tensor[tf.newaxis, ...]example
Can you explain that why my confusion matrix has shown 3 rows 3 columns. Although I have 2 classes. Evaluating model... | | # | 624 Elapsed Time: 0:00:28 Processed 625 images Saving confusion matrix... Confusion Matrix: [[199. 35. 0.] [ 43. 347. 0.] [ 45. 76. 0.]]
category ... [email protected]0 NORMAL ... 0.850427 1 PNEUMONIA ... 0.889744 [2 rows x 3 columns] Done!
It should show 2 row 2 col
I don't know why the third one is there
U can check your inputs it may have a third label or something (I don't really know 🥲)
Can you please let me know how it has calculated precision and recall for separate class, like separate precision for normal class and separate precision for pneumonia class??
Do you mean the equation?
Keep in mind I am not the creator of the script so I can't explain the code clearly
But the equations can be explained in this picture
