Classification: Data-preprocessing for Much Higher Accuracy and Confidence Level

[kcfdaniel]

Hi Dusty, I'm using a Jetson Nano 2GB, and using the classification pipeline. I was struggling with the accuracy and confidence level for quite some time. I'm trying to classify 3 classes, and most of the time, it just got it wrong, or sometimes right, with low confidence. I knew that it was NOT related to training, coz after training, it shows Acc@1 97.xx.

I once suspected that it was the model conversion's issue, but there's almost nothing I could do about it.

At the end I reckoned that the data-preprocessing for the inferencing data and the training data might be different, so I tried to resize and crop the center of the image before feeding it to the network, then things improves DRASTICALLY!!!

This is what I changed to imagenet.py

...

# process frames until the user exits
while True:
  # capture the next image
  img_input = input.Capture()

  img_intermediate = jetson.utils.cudaAllocMapped(width=img_input.width/img_input.height*224, 
                                         height=224, 
                                         format=img_input.format)
  
  # rescale the image (the dimensions are taken from the image capsules)
  jetson.utils.cudaResize(img_input, img_intermediate)

  crop_roi = ((img_intermediate.width - 224)/2, 0, 224 + (img_intermediate.width - 224)/2, 224)
  img = jetson.utils.cudaAllocMapped(width=224,
                                         height=224,
                                         format=img_intermediate.format)
  
  jetson.utils.cudaCrop(img_intermediate, img, crop_roi)

  # classify the image
  class_id, confidence = net.Classify(img)

...

Not sure why your S3E3 video was working so well, but mine needed a little tweak, I'd like to know as well. Was it because of different versions of the code, or different machines (Jetson Nano vs something else)?