snapml-templates

This repository provides training notebooks for Lens Studio templates powered with Snap ML. Each notebook allows to train a model which then can be brought into Lens Studio project.

Lens Studio download page

SnapML overview

Templates

Style Transfer

Allows you to train Style Transfer model based on provided image.

• Notebook
• Dataset: COCO dataset
• Template Guide

Classification

Provides an example of binary classification

• Notebook
• Dataset: CelebA
• Template Guide

Object Detection

Provides an example of the car detection

• Notebook
• Dataset: COCO
• Template Guide

Custom Segmentation

Provides an example of pizza segmentation

• Notebook
• Dataset: COCO
• Template Guide

Keyword Detection

Provides an example of training a model that classifies spectrogram images generated from audio.

• Notebook
• Dataset: SpeechCommands
• Template Guide

Image-to-Image Translation

Demonstrates how to train and compress popular image-to-image networks like CycleGAN and Pix2Pix so that we could achieve real time performance on mobile devices.

• Notebook
• Template Guide (This model is compatible with Style Transfer Template)

Multi Class Classification with Quantization

Demonstrates how to train an image classification models with Keras and TFLite model maker and quantize them using TensorFlow

• Notebook
• Template Guide

Multi-Object Detection

Demonstrates how to train a multi-object detection model using custom datasets annotated by ourselves with a step-by-step example on berries detection.

• Notebook
• Template Guide

SnapAR Global Lensathon - SnapML Workshop

Educational walkthrough of how to train your own image classifier from scratch and making it SnapML compatible. This is intended for educational purposes only. Please review relevant dataset licenses prior to usage.

• Notebook

Optical Character Recognition

Through Snap’s partnership with OpenCV, we are bringing to you a training notebook that allows you to train ML Models that can be later brought into Lens Studio and used to recognize text in the camera view.

• Inference Notebook
• Training Notebook (optional)
• Template Guide

Real-Time Neural Rendering

The template demonstrates how to render 3D assets real-time on device on Snapchat, using 2D images captured with a phone. There are also notebooks with training code to generate any 3D asset, and helpful guides covering how to deploy these models into a Lens.

This is a state-of-the-art model based on a research paper from Snap’s Creative Vision Research team that was accepted into CVPR this year, one of the most important and major computer vision and ML conferences in the world, where we also presented a live demo. This model and template in Lens Studio showcases how Snap’s research becomes an applied project that any developer can use.

• Inference Notebook
• Guide
• Lens Studio Project

License

A license file is included with each folder project.