Evaluation of a Yolov3 architecture after training returns mAP: 0

[Edwin-Aguirre92]

Hi,

I've re-trained a Yolov3 architecture to detect a specific object( yellow crane) for 50 experiments(takes about 28 hrs to train), the dataset used is 112 images to train and 28 images to evaluate . Further, I have trained a Yolov3 architecture using imageAI with a batch size of 4 in the past, which yielded a mAP of 34.5. However, this time I decided to use a batch size of 2,and after the 50 experiments were done, I checked the accuracy all the models trained, and all of them returned a mAP of 0.

The code used to train the Yolov3 architecture is given by this :

trainer = DetectionModelTrainer() trainer.setModelTypeAsYOLOv3() trainer.setDataDirectory(data_directory=image_folder+dataset_folder) trainer.setTrainConfig(object_names_array=["yellow crane"], batch_size=2, num_experiments=50, train_from_pretrained_model="pretrained-yolov3.h5") trainer.trainModel()

The code to check the accuracy of the model is given by this :

trainer = DetectionModelTrainer() trainer.setModelTypeAsYOLOv3() trainer.setDataDirectory(data_directory=image_folder+dataset_folder) metrics = trainer.evaluateModel(model_path="Dataset_stratified_no_gaussian\models", json_path="Dataset_stratified_no_gaussian\json\detection_config.json", iou_threshold=0.5, object_threshold=0.3, nms_threshold=0.5) print(metrics)

This is what I get when I run the code above(code to check the accuracy):

Evaluation samples: 28 Using IoU: 0.5 Using Object Threshold: 0.3 Using Non-Maximum Suppression: 0.5 yellow crane: 0.0000 mAP: 0.0000

WARNING:tensorflow:No training configuration found in the save file, so the model was not compiled. Compile it manually. Model File: Dataset_stratified_no_gaussian\models\detection_model-ex-036--loss-0007.861.h5

I'm wondering if this issue is because the batch size was change to 2 rather than 4? As I have stated I have trained Yolov3 models previously using ImageAI and I had no issues. I can also see that after every experiment is done the loss function value decreases. Lastly, I went back to the labelImg software and noticed that one picture had a double labeled called "yellow crane" and not sure if that is the issue ?

Thank you

[TheBeastCoding]

Try the following and let me know if it fixes: After training a model, delete the workspace cache: (ex. !rm -r .../<name_of_workspace>/cache/). During the evaluation step, ImageAI will recreate this folder along with the .pkl files. Between model training and evaluation, sometimes the file becomes "corrupted", which will give a default mAP of 0.0. Deleting the cache after training resolved this issue every time it occurred for me.

[TheBeastCoding]

Also, 50 experiments for a training size of 50 and taking over a day is way too long. Consider using the free GPU resources provided by Google Collab: https://colab.research.google.com/, which could reduce your training time to a few hours.