opentera-webrtc
opentera-webrtc copied to clipboard
introlab
WebRTC JavaScript, C++ and Python libraries and signaling server for the OpenTera project with hardware acceleration with GStreamer
OpenTera - WebRTC
WebRTC is a standard for real-time audio/video/data communication and is mostly used in web browsers. For the OpenTera micro-service architecture, we wanted to use Google's native WebRTC implementation for robots so we would have more control on the streams and develop a framework that is optimized for robot teleoperation. We also wanted to use hardware acceleration if possible on embedded platforms.
This project provides:
- A signaling server written in Python that can be used as a stand-alone server or embedded in our OpenTera Teleoperation Micro-Service
- A native libwebrtc wrapper that is used as a C++ base library for clients using our signaling server.
- GStreamer video codec factories for libwebrtc to enable both software and hardware encoders/decoders if available.
- Possiblity to force usage of a particular codec with VideoStreamConfiguration
- Python 3 bindings that are used as a Python module for clients using our signaling server.
- A JavaScript library That is used for web client developement and teleoperation interface using our signaling server.
License
By default, libwebrtc is built with non-free codecs. See webrtc-native-build to build without them.
- Apache License, Version 2.0
- For GStreamer components, please read this.
API Documentation
API Documentation is available here.
How to Contribute
Please read the Code of Conduct and CONTRIBUTING files.
Dependencies
- Most dependencies are part of the project with git submodules in the opentera-webrtc-native-client/3rdParty directory.
Build Instructions
- Please see Build Instructions.
Examples
C++
- data-channel-client
- stream-client
Python
- data-channel-client
- stream-client
Javascript
- data-channel-client
- stream-client
- stream-data-channel-client
Hardware Acceleration with GStreamer
Hardware acceleration depends on the platform you are using to build opentera-webrtc. The following table summarizes supported hardware and GStreamer plugins:
| Platform | VP8 | VP9 | H.264 |
|---|---|---|---|
| Jetson TX2/Nano | nvv4l2vp8enc, nvv4l2decoder | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder |
| Jetson Xavier NX | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder |
| Jetson AGX Xavier | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder |
| Jetson Orin / Orin Nano / Orin NX | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder |
| Raspberry Pi 4 | X | X | v4l2h264enc, v4l2h264dec |
| VA-API | vaapivp8enc, vaapivp8dec | vaapivp9dec | vaapih264enc, vaapih264dec |
| Apple Media | X | X | vtenc_h264, vtdec |
Note that VP9 encoding is under development, see issue 117.
ROS
- The C++ library and Python bindings are used in our opentera_webrtc_ros ROS package. RGB images published in ROS topics can be easily used for streaming in a RTCPeerConnection using our signaling server. Hardware acceleration and codec selection can be used if available.
Frontend
- The JavaScript library is used for our opentera WebRTC teleoperation frontend.
Authors
- Marc-Antoine Maheux (@mamaheux)
- Dominic Létourneau (@doumdi)
- Philippe Warren (@philippewarren)
- François Michaud (@michaudf)
Contributors
- Cédric Godin (@godced)
- Ian-Mathieu Joly (@joli-1801)
Sponsor
IntRoLab - Intelligent / Interactive / Integrated / Interdisciplinary Robot Lab
introlab