detecto
detecto copied to clipboard
alankbi
To normalize or not to normalize
On the pre-trained model (without any finetuning or custom training), the accuracy is terrible if images passed into the model are normalized. This will probably require some investigation as to how to properly apply normalization to images, if you're supposed to at all.
The temporary workaround right now is to disable normalizing when just using the default weights.
I think the torchvision FasterRCNN model already has normalization built in, unrelated to this, but my image Tensor was in the wrong format and I was getting errors in the following code while calling model.fit with detecto. It seems the model has normalization built in.
https://github.com/pytorch/vision/blob/883f1fb01a8ba0e1b1cdc16c16f2e6e0ef87e3bd/torchvision/models/detection/faster_rcnn.py#L227
https://github.com/pytorch/vision/blob/883f1fb01a8ba0e1b1cdc16c16f2e6e0ef87e3bd/torchvision/models/detection/transform.py#L120
Could this be causing this issue? Although wouldn't normalizing twice to the same values cause nothing to happen?
Would you be able to provide more specifics about what your error is? As for the normalization, it does seem like there's some built-in normalization going on. With the default model, normalizing twice causes very poor results (which is why I have it disabled), but it looks about the same when using a custom-trained model.
I don't have an error anymore, was just explaining how I stumbled on that code. Saw this open issue and wanted to make sure you were aware that the model has normalization built in.