TimeWarp - 在线视频背景替换

Replacing Real-Time the High-Resolution Meeting Background Freely.

✨ Features

• Real-Time High-Resolution Background Matting,  ;
• Replacing our background in meeting screen;
• Replace background with any size photo you want;
• Cross mutil-platform, like zoom plugin, web-browser, miniprogram in PC, mobile and iPad, any platform with web service;
• IFLYTEK voice translation between Chinese and English;
• ...

Overview

• Updates
• Download
  • Model / Weights
  • Video / Image Examples
  • Datasets
• Build
• Demo
  • Beta
  • Scripts
  • Notebooks
• Usage / Documentation
• Training
• License
• Reference
• Community

Updates

• [Jan 09 2021] PhotoMatte85 dataset is now published.
• [Dec 21 2020] We updated our project to MIT License, which permits commercial use!

Download

Model / Weights

• Download model / weights

Video / Image Examples

• HD videos (by Sengupta et al.) (Our model is more robust on HD footage)
• 4K videos and images

Datasets

• VideoMatte240K (Coming soon)
• PhotoMatte85

🔨 Build

1. setup enviroment

# git clone https://github.com/Charmve/TimeWarp.git
# cd TimeWarp

# pip install -r requirements.txt

2. download dataset and model Download

3. run test code

• Video

# cd TimeWarp
# bash test_video.sh

• Image

# cd TimeWarp
# python3 test_image.py

Demo

🎧 Beta

demo Real-Time High-Resolution Background Matting^[1]

Have a Try !

➡️ Web Demo by Gradio: Matting your own images from your browser.

Scripts

We provide several scripts in this repo for you to experiment with our model. More detailed instructions are included in the files.

• inference_images.py: Perform matting on a directory of images.
• inference_video.py: Perform matting on a video.
• inference_webcam.py: An interactive matting demo using your webcam.

Notebooks

Additionally, you can try our notebooks in Google Colab for performing matting on images and videos.

• Image matting (Colab)
• Video matting (Colab)

Virtual Camera

We provide a demo application that pipes webcam video through our model and outputs to a virtual camera. The script only works on Linux system and can be used in Zoom meetings. For more information, checkout:

• Webcam plugin

Web Demo

Developers in the community has helped us build a web demo. See Community Projects section below.

Usage / Documentation

You can run our model using PyTorch, TorchScript, TensorFlow, and ONNX. For detail about using our model, please check out the Usage / Documentation page.

中文解读 ->

• 单块GPU实现4K分辨率每秒30帧，华盛顿大学实时视频抠图再升级，毛发细节到位
• 整个世界都是你的绿幕：这个视频抠图换背景的方法着实真假难辨

Training

Training code will be released upon acceptance of the paper.

☕ Processing

gantt
        dateFormat  YYYY-MM-DD
        title TimeWarp Development Processing
        section image matting
        architecture design          :done,    des1, 2020-11-06,2020-12-20
        video matting                :active,  des2, 2021-01-07, 3d
        chrome extension plus        :         des3, after des2, 7d
        video meeting architecture   :         des4, after des3, 5d

• More update processing details, please click HERE to see CHANGLOG.

🔔 Update Reminder

Subscribe -->

Business ✉️ email: [email protected]

🍮 Community

• GitHub discussions 💬 or issues 💭

• QQ Group: 697948168 (password：TimeWarp)

• Email: yidazhang1#gmail.com

🔗 Reference

[1] Real-Time High-Resolution Background Matting. Shanchuan Lin*, Andrey Ryabtsev*, Soumyadip Sengupta. [GitHub] | [arXiv] | [data]

[2] Background Matting: The World is Your Green Screen. Soumyadip Sengupta, Vivek Jayaram, Brian Curless, et.al. CVPR 2020. [GitHub] | [arXiv]

[3] MODNet: Is a Green Screen Really Necessary for Real-Time Portrait Matting? [GitHub] | [arXiv]

[4] High-Resolution Deep Image Matting. Haichao Yu, Ning Xu, Zilong Huang. [GitHub] | [arXiv]

[5] End-to-end Animal Image Matting. Jizhizi Li, Jing Zhang, Stephen J. Maybank. [GitHub] | [arXiv].

[6] BGMv2-webcam-plugin-linux. https://github.com/andreyryabtsev/BGMv2-webcam-plugin-linux

[7] Chrome-extensions-samples. https://github.com/GoogleChrome/chrome-extensions-samples

[8] Zoom Video Ingester. https://github.com/harvard-dce/zoom-recording-ingester

[9] Zoom Recording Downloader. https://github.com/ricardorodrigues-ca/zoom-recording-downloader

[10] PaperswithCode. Image Matting. https://paperswithcode.com/task/image-matting

📜 License

This work is licensed under the Creative Commons Attribution NonCommercial ShareAlike 4.0 License.

💝 Acknowledge

Thanks for PeterL1n, et al.

Feel free to ask any questions, open a PR if you feel something can be done differently!

🌟Star this repository🌟

Created by Charmve & maiwei.ai Community | Deployed on Gradio.app