models icon indicating copy to clipboard operation
models copied to clipboard

Confusion Matrix

Open Annieliaquat opened this issue 3 years ago • 17 comments

How to find the confusion matrix for ssd-mobnet model?

Annieliaquat avatar Jul 23 '22 06:07 Annieliaquat

I am currently working on the same stuff but not success yet I am trying to following this link and this

hope it can help

leon5354 avatar Jul 25 '22 17:07 leon5354

I am currently working on the same stuff but not success yet I am trying to following this link and this

hope it can help

I have tried with both this links but they are not working.

Annieliaquat avatar Jul 26 '22 18:07 Annieliaquat

I am currently working on the same stuff but not success yet I am trying to following this link and this hope it can help

I have tried with both this links but they are not working.

What happened when it is not working? What are you trying to do and how do you want it to work? I assumed you trained a model and want to use it for prediction and check the confusion matrix? What tensorflow version are you using? did you tried both file in the link for tf1 or tf2?

leon5354 avatar Jul 27 '22 03:07 leon5354

I am currently working on the same stuff but not success yet I am trying to following this link and this hope it can help

I have tried with both this links but they are not working.

What happened when it is not working? What are you trying to do and how do you want it to work? I assumed you trained a model and want to use it for prediction and check the confusion matrix? What tensorflow version are you using? did you tried both file in the link for tf1 or tf2?

Both this links need prediction.csv file to make confusion matrix. yes I have trained my model and have done prediction, just last thing I want is confusion matrix. Have you created the confusion matrix please how let me know

Annieliaquat avatar Jul 27 '22 06:07 Annieliaquat

just use this link it does not require a prediction file

leon5354 avatar Jul 27 '22 13:07 leon5354

What is this detection_box and ground truth.?? Ground_truth is ofcourse the labelled data, what will be the detection_box??

On Wednesday, July 27, 2022, leon5354 @.***> wrote:

just use this link https://github.com/svpino/tf_object_detection_cm/blob/master/confusion_matrix_tf2.py it does not require a prediction file

— Reply to this email directly, view it on GitHub https://github.com/tensorflow/models/issues/10719#issuecomment-1196784272, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKIIFWL7BO3RCBYD7XJTGFDVWE4TDANCNFSM54NPQNQQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Best Regards Bibi Qurat-ul-Ain

Annieliaquat avatar Jul 27 '22 14:07 Annieliaquat

just use this link it does not require a prediction file

?????

Annieliaquat avatar Jul 29 '22 13:07 Annieliaquat

just use this link it does not require a prediction file

can you just let me know on how to get this file --detections_record=testing_detections.record ? it is from the link you provided

Annieliaquat avatar Jul 29 '22 13:07 Annieliaquat

tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') -tfrecord you used to train the model tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') -where you want it to be outputed tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') -the model you want to use to generate the confusion matrix tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') -the pbtxt file of this model tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') -do you want the prediction of images to be save true or false tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images') -if true where you want it to be save

Example of how to use the script: dont know what platform you use, just give you an example you to replace whatever suits you

%cd 'path to the folder of your script' !python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt

just install whatever you needed when you train the model to prevent any error

leon5354 avatar Jul 29 '22 16:07 leon5354

tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') -tfrecord you used to train the model tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') -where you want it to be outputed tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') -the model you want to use to generate the confusion matrix tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') -the pbtxt file of this model tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') -do you want the prediction of images to be save true or false tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images') -if true where you want it to be save

Example of how to use the script: dont know what platform you use, just give you an example you to replace whatever suits you

%cd 'path to the folder of your script' !python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt

just install whatever you needed when you train the model to prevent any error

I have got this error after I wrote the code. This is the code !python confusion_matrix_tf2.py --input_tfrecord_path=/content/drive/MyDrive/DetectionApi/workspace/annotations/test.record --output_path=/content/drive/MyDrive/DetectionApi/workspace/annotations/confusion_matrix.csv --inference_graph=/content/drive/MyDrive/DetectionApi/workspace/models/exported-model/saved_model/ --class_labels=/content/drive/MyDrive/DetectionApi/workspace/annotations/label_map.pbtxt

This is the error. Please help me out. image

Annieliaquat avatar Jul 29 '22 18:07 Annieliaquat

tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') -tfrecord you used to train the model tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') -where you want it to be outputed tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') -the model you want to use to generate the confusion matrix tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') -the pbtxt file of this model tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') -do you want the prediction of images to be save true or false tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images') -if true where you want it to be save

Example of how to use the script: dont know what platform you use, just give you an example you to replace whatever suits you

%cd 'path to the folder of your script' !python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt

just install whatever you needed when you train the model to prevent any error

Can you see the error I have posted

Annieliaquat avatar Jul 30 '22 10:07 Annieliaquat

tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') -tfrecord you used to train the model tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') -where you want it to be outputed tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') -the model you want to use to generate the confusion matrix tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') -the pbtxt file of this model tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') -do you want the prediction of images to be save true or false tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images') -if true where you want it to be save Example of how to use the script: dont know what platform you use, just give you an example you to replace whatever suits you %cd 'path to the folder of your script' !python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt just install whatever you needed when you train the model to prevent any error

Can you see the error I have posted

I have no idea, maybe double-check your file to me it seems like your input dimension and model dimension is different

leon5354 avatar Jul 30 '22 13:07 leon5354

tf.compat.v1.flags.DEFINE_string('input_tfrecord_path', None, 'Input tf record path') -tfrecord you used to train the model tf.compat.v1.flags.DEFINE_string('output_path', None, 'Path to the output CSV.') -where you want it to be outputed tf.compat.v1.flags.DEFINE_string('inference_graph', None, 'Path to the inference graph with embedded weights.') -the model you want to use to generate the confusion matrix tf.compat.v1.flags.DEFINE_string('class_labels', None, 'Path to classes.pbtxt file.') -the pbtxt file of this model tf.compat.v1.flags.DEFINE_string('draw_option', "False", 'Whether or not to save images with boxes drawn') -do you want the prediction of images to be save true or false tf.compat.v1.flags.DEFINE_string('draw_save_path', None, 'If using draw_option, where to save images') -if true where you want it to be save Example of how to use the script: dont know what platform you use, just give you an example you to replace whatever suits you %cd 'path to the folder of your script' !python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt just install whatever you needed when you train the model to prevent any error

Can you see the error I have posted

I have no idea, maybe double-check your file to me it seems like your input dimension and model dimension is different

Now the error

Traceback (most recent call last): File "confusion_matrix_tf2.py", line 306, in tf.compat.v1.app.run() File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/platform/app.py", line 36, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 308, in run _run_main(main, args) File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 254, in _run_main sys.exit(main(argv)) File "confusion_matrix_tf2.py", line 297, in main confusion_matrix = process_detections(input_tfrecord_path, model, categories, draw_option, draw_save_path) File "confusion_matrix_tf2.py", line 125, in process_detections input_tensor = tf.convert_to_tensor(img) File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py", line 102, in convert_to_eager_tensor return ops.EagerTensor(value, ctx.device_name, dtype) ValueError: Attempt to convert a value (<PIL.Image.Image image mode=RGB size=1064x720 at 0x7F2F5B128110>) with an unsupported type (<class 'PIL.Image.Image'>) to a Tensor.

Annieliaquat avatar Jul 30 '22 18:07 Annieliaquat

Call Park Jed or Christine. They stole my latest numberOn Sat, Jul 23, 2022 at 2:11 AM Sania @.***> wrote: How to find the confusion matrix for ssd-mobnet model?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

akaye47 avatar Jul 30 '22 22:07 akaye47

I’m not confused.You’re just assholesOn Sat, Jul 30, 2022 at 6:30 PM Andrea Kulikowski @.> wrote:Call Park Jed or Christine. They stole my latest numberOn Sat, Jul 23, 2022 at 2:11 AM Sania @.> wrote: How to find the confusion matrix for ssd-mobnet model?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

akaye47 avatar Oct 11 '22 06:10 akaye47

!python confusion_matrix_tf2.py --input_tfrecord_path=/path/test.record --output_path=/path/confusion_matrix.csv --inference_graph=/path/saved_model/ --class_labels=/path/label_map.pbtxt

Hello! I'm facing this error

ValueError: Python inputs incompatible with input_signature: inputs: ( [[[[186 181 184 255] [177 172 176 255] [155 150 154 255] ... [ 50 48 76 255] [ 49 48 78 255] [ 48 48 78 255]]

[[186 181 184 255] [178 173 178 255] [158 153 156 255] ... [ 49 48 76 255] [ 48 48 77 255] [ 47 47 77 255]]

[[186 181 184 255] [179 174 178 255] [160 155 159 255] ... [ 49 48 76 255] [ 48 48 78 255] [ 47 47 78 255]]

...

[[ 70 67 68 255] [ 71 68 69 255] [ 70 68 68 255] ... [141 135 129 255] [141 135 127 255] [141 135 127 255]]

[[ 71 69 69 255] [ 71 69 69 255] [ 72 70 71 255] ... [140 135 129 255] [141 135 127 255] [141 135 127 255]]

[[ 72 70 70 255] [ 73 71 71 255] [ 73 71 71 255] ... [140 136 129 255] [141 135 128 255] [141 135 127 255]]]]) input_signature: ( TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor')).

can you please help me to fix it..

AmiraShekhi avatar Nov 25 '22 17:11 AmiraShekhi

2023-01-17 21:11:30.683137: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.8/dist-packages/cv2/../../lib64:/usr/lib64-nvidia 2023-01-17 21:11:30.683250: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.8/dist-packages/cv2/../../lib64:/usr/lib64-nvidia 2023-01-17 21:11:30.683271: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. /usr/local/lib/python3.8/dist-packages/scipy/init.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.24.1 warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}" /content/drive/Shareddrives/Nayannath70/training_demo/annotations/test.record Loading model... 2023-01-17 21:11:33.549202: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. Evaluating model... | | # | 41 Elapsed Time: 0:00:16 Traceback (most recent call last): File "confusion_matrix_tf2.py", line 303, in tf.compat.v1.app.run() File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/app.py", line 36, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run _run_main(main, args) File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main sys.exit(main(argv)) File "confusion_matrix_tf2.py", line 294, in main confusion_matrix = process_detections(input_tfrecord_path, model, categories, draw_option, draw_save_path) File "confusion_matrix_tf2.py", line 126, in process_detections detections = model(input_tensor) # Run model inference File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 732, in _call_attribute return instance.call(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/polymorphic_function/function_spec.py", line 698, in convert_inputs_to_signature raise ValueError("Python inputs incompatible with input_signature:\n" ValueError: Python inputs incompatible with input_signature: inputs: ( [[[199 199 199 ... 188 187 187] [199 199 199 ... 188 187 187] [199 199 199 ... 188 187 187] ... [205 205 206 ... 174 174 174] [205 205 206 ... 174 174 174] [205 205 206 ... 174 174 174]]]) input_signature: ( TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor')).

can you please help me to fix it..

nayannath70 avatar Jan 17 '23 21:01 nayannath70