Add FaceSearch to UI

[tuliocasagrande]

Hey folks,

This PR is to add the FaceSearch functionality to the webapp.

I started using the Celebrities component as baseline, only changing a few lines with the operation name. Now my understanding is that Elasticsearch and its corresponding Lambda function should carry the rest of the work. Is that correct?

I'm opening the PR this early to get feedback. Thank you!

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[ianwow]

Yes, .../source/consumers/elastic/lambda_handler.py consumer is already setup to save Rekognition face search results into Elasticsearch. All you need to do is add a tab to the UI (views/Analysis.vue) and create a FaceSearch component under components/. I would recommend modeling it after components/FaceDetection.vue.

[ianwow]

Sorry, I accidentally closed this issue. Reopened.

[tuliocasagrande]

I would recommend modeling it after components/FaceDetection.vue.

Hey Ian, I considered modeling it after components/FaceDetection.vue, but GetFaceSearch can only return the default facial attributes (BoundingBox, Confidence, Landmarks, Pose, and Quality) and not (AgeRange, Beard, Emotions, Eyeglasses, EyesOpen, ...) as GetFaceDetection does. More information: API_FaceDetail and API_GetFaceSearch.

Because of that, I started modeling it after components/Celebrities.vue. In fact, so far only some identifiers were changed:

$ diff webapp/src/components/FaceSearch.vue webapp/src/components/Celebrities.vue
98c98
<     name: "FaceSearch",
---
>     name: "Celebrities",
117c117
<         operator: 'face_search',
---
>         operator: 'celebrity_detection',

What I'm designing now is what information we should render in the UI. I'm considering something with the FaceId and ExternalImageId.

[tuliocasagrande]

I finished the FaceSearch component. My proposal is the UI to show Person{index_number} in case of an unknown face and ExternalImageId in case of a known one.

This behavior is going to be better used if Recommendations for Facial Recognition Input Images were followed. More specifically:

If you want to track that these face instances belong to the same individual, consider using the external image ID attribute if there is only one face in the image being indexed. For example, five images of John Doe can be tracked in the collection with external image IDs as John_Doe_1.jpg, … John_Doe_5.jpg

The only difference is that instead of "guessing" the suffix format (_1.jpg, _5.jpg) to remove, we can suggest using the same canonical format, only John_Doe in the given example. I can document this in the README or IMPLEMENTATION_GUIDE.

@ianwow What do you think?

[ianwow]

There are 2 problems with using ExternalImageId.

1. ExternalImageId is optional, so matching faces may not always have that property.
2. ExternalImageId can be an arbitrary string, so users may not assign it a name that makes sense to surface in the UI. Even if we document a desired name format there will be many cases where users will not have read those docs.

I think the most generally applicable case is for the UI to show the FaceId by default. We could always provide a radio button that allows users to toggle between showing FaceId and ExternalImageId.

[tuliocasagrande]

Good point, @ianwow.

I changed the code to show the FaceId by default. Maybe the radio button could be a separate PR.

[ianwow]

The build script in your branch is failing. I know you didn't make any changes to it, so it's probably broken the branch that you forked from. I think we'll need to change the base on this PR, but I'm not sure what will work, yet. I look at this next week.

[ianwow]

@tuliocasagrande Your PR base needs to be updated. Could you please pull the latest development branch, rebase your facesearch branch on that and resubmit your PR? Like this:

# Add the official repo as a remote
git remote -v
git remote add mie_official https://github.com/awslabs/aws-media-insights-engine.git
git remote -v
# Fetch the latest version of the development branch
git checkout development
git fetch mie_official development:development2
git checkout facesearch
# merge with development and resolve all conflicts
git rebase mie_official/development
# now push your changes to origin
git push -u origin facesearch
# finally submit a PR via Github's webui

[tuliocasagrande]

Rebased with awslabs:development.