[brainstorm] Pull ideas from Digikam

[stalker314314]

I just saw article about Digikam using DLib and face recognition. It seems that they also have their own pretrained model (maybe even same, didn't go to details), some clustering (KNN maybe?) and they have some usability improvements (remove face from group etc.), maybe some other interesting pieces (save faces to image, recognize dogs...). It would be useful to understand how they did it, maybe even steal something (ideas or code:D). You can use this issue as a way to analyze and dump/form ideas (not sure if this is best approach, close if you don't see this fit).

Here is relevant link: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ FYI, there is also HN discussion where I mentioned this project: https://news.ycombinator.com/item?id=23947398 here and here

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[matiasdelellis]

Yeah .. I saw it and thought exactly the same. :grimacing: Maybe next week I will try it, and take ideas .. :wink:

[escoand]

Found an interesting comparison and summary of all the well known face detection algorithms: https://www.learnopencv.com/face-detection-opencv-dlib-and-deep-learning-c-python/

[matiasdelellis]

Hi @escoand Thank you for the link. I had read it like so many others. :joy:

This particular article makes a very good comparison between detectors. It may seem that OpenCV DNN works better than Dlib (Just for the face size limitation), but keep in mind that the face recognition is a completely different task and most articles recommend Dlib models and many articles use a combination of both methods.

Although we are open to receiving new models, and the application is quite prepared for it, today we believe we have a very good set.

Now we are focused on improving the face clustering system, face suggestions for persons, and about how to improve of results. The idea is to test other applications, to see how they work, and see what we can implement to improve. :grimacing:

[escoand]

@matiasdelellis ok, than I would like to ask if digiKam and this Nextcloud add-on are using the same tagging schema? So you could use either software and switch some day.

[matiasdelellis]

Ok.. It was hard to find them, but the workflow is this.. :sweat_smile:

1. Go tho the persons tab
2. Click on Scan the collection looking for faces button. NOTE: This point is magic. I analyze the 433 images of TBBT in 38 seconds. Wow.. :tada:
3. It shows you the 657 faces that found, and you can label persons one by one as the image shows. NOTE: Our application finds 1456, but of course it is 68 times slower.
4. Go to unknown, and tag some faces, for example 20 Sheldon's.
5. Change to the option "Recognize faces" and again click on Scan the collection looking for faces button again. NOTE: Again very fast.
6. Go to sheldon.. and accept the faces one by one again.
7. Repeat 5 and 6 until infinity of time. :sweat_smile: and always one face at a time... :disappointed:

Well... I only got 68 sheldon's and many more were untaggeds. IMHP, except the speed of initial analysis, it is quite disappointing . :disappointed: The approach of PR https://github.com/matiasdelellis/facerecognition/pull/262 is much more advanced.

[matiasdelellis]

p.s.: Do not analyze the database or the tools used yet.

[escoand]

I've also tested digiKam. The face detection worked out quite well for me, but the face recognition was also nearly useless. I had the feeling with every tagged face the recognition getting worse, like the AI is unable to cope with too much information about one face.

But in contrary to the announcement it's really disappointing. Already wondered if I use the new DNN feature.

[escoand]

One extra point: digiKam creates EXIF/XMP tags and keywords. This could be search for by any software and would be a nice solution additional to the information in the database.

[matiasdelellis]

Hi @escoand

One extra point: digiKam creates EXIF/XMP tags and keywords. This could be search for by any software and would be a nice solution additional to the information in the database.

I was just thinking about the opposite case.

1. Upload photos data from the phone.
2. Do the analysis with our application.
3. Write XMP reginons to photos with this application.
4. Take the analysis in any other application with EXIF/XMP.

But of course the opposite case is also interesting. Especially if the XMP region has a name, we should trust it. :wink:

However, we are limited by Nextcloud's infrastructure. See https://github.com/nextcloud/photos/issues/226#issuecomment-595744332 to know more or less the actual situation.

[escoand]

@matiasdelellis Anyway, some kind of standard tagging schema would be helpful. Is this plugin already saving the results into the images?