Live Face Recognition App

This is a Streamlit App that allows you to recognize faces in a live webcam video stream. Check out the app on the Streamlit Community Cloud by clicking on the badge below:

👀 Example

As an example, instead of using a webcam I fed a snippet of BigBangTheory into the app and it recognized the characters in real-time:

🚀 Usage

Run the App on Streamlit Community Cloud

You can run my app in your Browser using the Streamlit Community Cloud by clicking on the badge below:

Locally on your machine (Tested on MacOS, Linux)

You can run the app locally on your machine by cloning this repository and running the following commands:

git clone https://github.com/Martlgap/livefaceidapp.git
cd livefaceidapp
pip -m venv venv
source venv/bin/activate
pip install -r requirements.txt
streamlit run main.py

Depending on platform and hardware you need to install onnxruntime, onnxruntime-gpu, or onnxruntime-silicon.

Locally on server (Tested on Linux)

If you want to run the app on a server in your local network and would like to access it from another machine in the same network, you can use the ssl-proxy plugin of suyashkumar:

wget https://github.com/suyashkumar/ssl-proxy/releases/download/v0.2.7/ssl-proxy-linux-amd64.tar.gz

gzip -d ssl-proxy-linux-amd64.tar.gz
tar -xvf ssl-proxy-linux-amd64.tar

./ssl-proxy-linux-amd64 -from 0.0.0.0:8502 -to 0.0.0.0:8501

After that you are able to access the app in your browser via https://your-server-ip:8502

🖥️ Streaming Servers

For beeing able to use the app on the Streamlit Communnity Cloud, I set up a TWILIO TURN server with my credentials. Those are saved in the streamlit community cloud secrets, which is not included in this repository. If you want to use the app on your own server, you have to set up your own TURN server. You can find a detailed description on how to do that here.

You can set your own credentials by exporting them as environment variables:

export TWILIO_ACCOUNT_SID=your-twilio-account-sid
export TWILIO_AUTH_TOKEN=your-twilio-auth-token

⚙️ How it Works

A detailed description of the implementation can be found here:

🧠 Machine Learning Models

The app uses the following machine learning models:

• Face Detection: FaceMesh (MediaPipe)
• Face Recognition: MobileNetV2 Architecture, trained with MS1M dataset and ArcFace Loss using Tensorflow, converted to ONNX

📖 About

I developed this app during a project at the Chair of Human-Machine Communication. The goal was to compare live face recognition systems across different platforms.

📚 Resources

Please take a look at the following resources, which helped me a lot during the development of this app:

• Streamlit
• Streamlit Community
• Streamlit Webrtc
• ssl-proxy
• MediaPipe
• aiortc
• WebRTC
• WebRTC Samples
• Deployment
• Real-Time-Video-Streams

🪲 BUGS - KNOWN ISSUES - TODOS:

• [ ] Running the app on streamlit community cloud introduces a severe lag/delay in the video stream. This is also present when working with the app locally on a server. There should be a way to reduce the delay, either by setting the buffer size smaller or by using frame dropping.
• [ ] There is also sometimes a lag when running the app locally on your machine. This is due to several reasons and need to be investigated further.
• [ ] Aspect ratio bug on smartphone