mediapipe
mediapipe copied to clipboard
google-ai-edge
bject_detector.Dataset.from_pascal_voc_folder ignores negative/background images
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Linux
Python Version
3.10
MediaPipe Model Maker version
0.2.0
Task name (e.g. Image classification, Gesture recognition etc.)
Object detection
Describe the actual behavior
object_detector.Dataset.from_pascal_voc_folder()
Is not including negative/background images. These are images that have no object / bounding box in the XML. These are crucial to include for good object detection training. This differs with how tflite-model-maker works, where these images are included.
Describe the expected behaviour
Similar as with how it works in tflite-model-maker, images with no bounding box are included in the training, for example when the annotation file is as below:
<annotation verified="yes">
<folder>unprocessed_control</folder>
<filename>0b44ce15-aed7-4330-9988-2bbd17e69a7a.jpg</filename>
<path>0b44ce15-aed7-4330-9988-2bbd17e69a7a.jpg</path>
<source>
<database>Unknown</database>
</source>
<size>
<width>640</width>
<height>480</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
</annotation>
Standalone code/steps you may have used to try to get what you need
Use above XML with tflite-model-maker and see it being part of the training data, while with media pipe model maker only images with bounding boxes are included.
Other info / Complete Logs
No response
@willemnator, Would you please let us know the complete steps you have followed to reproduce the issue from our end?
i followed the object_detector customize from google tutorial with the addition of loading training data using from_pascal_voc_folder.
@willemnator,
Sorry for the delayed response, Are you still for the resolution on this issue or it has been resolved from your end? Thank you
@kuaashish This should be prio'd because at its current state you can't train a good Object detection model. To train a good model you have to be able to include background images. It is not solved.
@willemnator,
Could you please share detailed explanation which you have observed the difficulty to train a good object detection model? And We will let the team knows about the issue. Thank you!
In object detection training it doesn't allow for including negative/background images. With tflite-model-maker it does. That is it. More details are in the main post.
Hello @ahmadyan, @lzhang57,
Could you please look into this issue? Thank you!!
is it really true that the mediapipe fine-tuner ignores negative examples? this seems like quite a basic oversight. is anyone working to reproduce this issue?
Yeh at least when using from_pascal_voc_folder() - and it's still not fixed. I think google has moved all their resources to language models.
