Join clusters together...

[matiasdelellis]

I'm seeing how Google Photos works with my photos.. It is not magic either ... it shows less results, but without errors. The interesting thing is that the web version allows you to add photos to groups.

It only shows a notification ..

If you click theme:

If you say that it's the same person... just show another photo again..

Can not we do something similar? It should be used in combination of this https://github.com/matiasdelellis/facerecognition/issues/114 and https://github.com/matiasdelellis/facerecognition/issues/105

How to join them?

Add an facecognition_face_joins table..

userId	Face1	Face2
matias	1	2
matias	1	3
matias	1	...
matias	1	6775
matias	1	...
matias	n	2

That must be filled with this dialog..

Again... How to join them? :sweat_smile: At the end of the CreateClustersTask task, check if these photos are in the same group.

• If you are already in the same group, do nothing.
• If the faces are in different group, you have to try to join.
  • If the group to join is good. (in terms of https://github.com/matiasdelellis/facerecognition/issues/114) put all the photos together in the biggest one.
  • If the group is bad, just move the new face.

Note that the main group always remains stable.. The suggested groups can mutate among themselves, and between each analysis they can provide different faces to the main group. Although they are random (Which are not so much either), If they meet the conditions of https://github.com/matiasdelellis/facerecognition/issues/114, they will be correct photos. The important thing is to add the face that the user said they were the same person, but the suggestions may change..

How to get the suggestions? I do not know.. It can be a random face of the secondary groups, comparing them with a greater threshold/sensitivity?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[bjoern-tantau]

I'd keep the clusters separate from the persons. I think the clusters should just be a helper to add faces to a person. But ultimately the person-face relationship should be editable so that we can also remove a relationship in case it is wrong.

Google is also probably using these associations to improve their model. Can we do that as well?

[matiasdelellis]

Hi @bjoern-tantau Thanks for participating here. :smile:

This report is just a suggestion for the future, which we will evaluate later. Now we are focused on getting an initial version with the current development.

I'd keep the clusters separate from the persons.

To abstract this, we must add another complexity. However, for this to work, we must also trust in the stability of the clusters, which is still in evaluation, and we will confirm when we have more users.

Google is also probably using these associations to improve their model. Can we do that as well?

No. We have to work with standardized models.,

• https://github.com/davisking/dlib-models

Although we are aware of its limitations (Asian people, small children's, etc.), we do not have the capacity or experience to improve it.

[xkill]

I also suggest to use dlib learning process:

https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78

Basically:

1. Extract the faces from the images 1.1) Mark the images as processed
2. Create the clusters

If a user remove an image from a cluster, just mark it as removed, so future processing (clustering) of the library won't process it and, during the creation of the training images of dlib (SVM model) it won't be processed on this cluster/person.

If a user move an image, just mark it as part of a different cluster and regenerate the old cluster and the cluster trained data of dlib (SVM model)

So, next time, dlib will check the faces against the already trained models, if it does not match it should create a new cluster (like now).

Not sure if this comment should be on this ticket or on any other related tickets.

[matiasdelellis]

Hi @bjoern-tantau @xkill Although in a very different way, we are fixing this problem in #336. I would appreciate tests and comments.

The suggestions of new people will be left for another PR with #262

[matiasdelellis]

In the last release, we took an important step for this issue.. I hope to move forward soon with #262 :grimacing: :wink: