ml5-library icon indicating copy to clipboard operation
ml5-library copied to clipboard

Darknet image classifier always returns "nematode"

Open gohai opened this issue 3 years ago • 4 comments

When running the Darknet image classifier with the latest published version of ml5.js, I am always getting "nematode" as the label with the highest confidence, regardless of input image.

I've tested this with a webcam image, as well as with a static PImage of a bird.

The exact confidence varies slightly, indicating to me that the network is in principal running, and that the input is reaching it. But no matter the input, the output will always be "nematode".

Here's a sketch that demonstrate this issue (as of Feb 15 2022): https://editor.p5js.org/gohai/sketches/Wt1qPNPnq

The same sketch works with "MobileNet" and "DoodleNet" parameters to the image classifier initializer.

gohai avatar Feb 15 '22 07:02 gohai

Hi @gohai, I looked into this yesterday and I could not really figure this out. What I found was, that it always picks a certain item from our list of possible things. If I change nematode to my name it only displays my name. It also thinks that nematode is the most likely. I've seen the full list. I try to dig deeper into this. Maybe @shiffman, @bomanimc or @joeyklee have an idea?

tlsaeger avatar Feb 24 '22 10:02 tlsaeger

I was cleaning up some of our image-to-tensor conversions today and I noticed that we are going through pre-processing twice for darknet! There is a call to imgToTensor (which resizes and rescales) in the parent ImageClassifier. Then in darknet.js we are preprocessing again. It will end up with tiny values because we are dividing by 255 twice.

We may have the same problem in doodlenet too?

lindapaiste avatar May 22 '22 21:05 lindapaiste

I need to clean up my code before creating a PR but I just want to note that I do have this fixed! 🥳

lindapaiste avatar May 25 '22 18:05 lindapaiste

Happy hear - thank you @lindapaiste

gohai avatar May 26 '22 05:05 gohai