Understanding the XR Technology Stack

There are a myriad of options for developing immersive experience and the details complex. It seems as if biweekly, there are new SDKs, frameworks and technologies introduced to XR (Augmented, Virtual and Mixed Reality) developers.

Anyone new to the field or with a keen interest in learning may have the potential to be a great contributor but have no insights regarding the pipeline or how to approach building for XR. This repository seeks to consolidate information for current and aspiring XR developers.

Note: This is a resource from a lecture given November 17, 2021 for CS 460 Computer Graphics. This does not include cryptocurrencies/blockchain technologies. This is specifically geared towards XR, 3D, and gaming technologies

Contributing

To add to this resource, send a pull request. If you aren't familiar enough with git/Github, just open an issue.

Enjoy!

🍔

Frontend

A-Frame www.github.com/aframevr/aframe
Adobe Aero www.adobe.com/products/aero.html
AngularJS https://angularjs.org/
AR.js https://developer.vuforia.com/downloads/sdk
BabylonJS https://www.babylonjs.com/
JanusWeb https://github.com/jbaicoianu/janusweb
JavaScript https://www.javascript.com/
PrimroseVR https://www.primrosevr.com/
React XR https://www.npmjs.com/package/react-vr
Reality Composer https://apps.apple.com/us/app/reality-composer/id1462358802
RealityKit https://developer.apple.com/documentation/realitykit \

SDKs

ARCore https://developers.google.com/ar/
ARKit https://developer.apple.com/augmented-reality/
CameraKit SDK https://kit.snapchat.com/camera-kit
Cloud XR SDK https://developer.nvidia.com/nvidia-cloudxr-sdk
Easy AR https://www.easyar.com/
Google VR SDK (now Cardboard SDK) https://developers.google.com/vr/develop/unity/get-started-android
LightShip ARDK https://lightship.dev/
Lumin SDK https://developer.magicleap.com/downloads/lumin-sdk/
Mixed Reality Extension SDK https://github.com/Microsoft/mixed-reality-extension-sdk
MRTK https://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/
Normcore https://normcore.io/
Oculus SDK https://developer.oculus.com/
OpenVR SDK https://github.com/ValveSoftware/openvr
Photon https://www.photounengine.com/sdks
Snapdragon Spaces https://www.qualcomm.com/products/features/snapdragon-spaces-xr-platform/
Tobii XR SDK https://vr.tobii.com/sdk/
Vive Sense SDK https://developer.vive.com/resources/vive-sense/
VRTK https://www.vrtk.io/
Vuforia https://developer.vuforia.com/downloads/sdk
Wikitude https://www.wikitude.com/products/wikitude-sdk/
XRTK https://xrtk.io/

APIs

Advanced API https://vr.tobii.com/sdk/develop/unity/documentation/api-reference/
ARCore https://arvr.google.com/arcore/
Core API https://vr.tobii.com/sdk/develop/unity/documentation/api-reference/
Mozilla WebVR https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Fundamentals
Oculus https://dashboard.oculus.com/app/api
Open XR https://www.khronos.org/OpenXR/
WebAudio https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
threeJS https://github.com/mrdoob/three.js/
WebXR https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API

Protocols

SOAP https://www.w3.org/TR/soap/
WebRTC https://webrtc.org/
Websockets https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API

Databases

MySQL https://dev.mysql.com/doc/
NoSQL https://docs.oracle.com/en/database/other-databases/nosql-database/
PostgreSQL https://www.postgresql.org/docs/
VRML https://www.w3.org/MarkUp/VRML/
WebRTC https://webrtc.org/
Websockets https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
X3D https://www.web3d.org/x3d/what-x3d
xVRML https://sourceforge.net/projects/xvrml/

Backend

.NET https://docs.microsoft.com/en-us/dotnet/
C# https://docs.microsoft.com/en-us/dotnet/csharp/
C++ https://www.cplusplus.com/doc/tutorial/
Java https://docs.oracle.com/en/java/
lua https://www.lua.org/docs.html
Node.JS https://nodejs.org/en/docs/
Objective-C https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/Introduction/Introduction.html#//apple_ref/doc/uid/TP40011210
Python https://docs.python.org/3/
Ruby https://www.ruby-lang.org/en/documentation/
Swift https://www.swift.org/documentation/

Engines/Platforms

8th Wall Reality Engine https://www.8thwall.com/tutorials
Amazon Sumerian https://aws.amazon.com/sumerian/
Android https://developer.android.com/
Azure https://azure.microsoft.com/
iOS https://developer.apple.com/ios/
Lens Studio https://www.lensstudio.snapchat.com/
Omniverse https://developer.nvidia.com/nvidia-omniverse-platform/
PlayCanvas https://playcanvas.com/
Roblox https://www.roblox.com/create/
Spark AR https://sparkar.facebook.com/ar-studio/
Unity https://unity.com/ (also see Unity MARS)
Unreal https://www.unrealengine.com/
Windows Mixed Reality https://www.microsoft.com/en-us/mixed-reality/windows-mixed-reality

Appendix

                     ██████████████████████              
                   ██░░░░░░  ░░░░  ░░░░░░░░██ 
                 ██░░░░░░░░░░░░░░░░░░  ░░  ░░██          -- FRONT-END
               ██░░░░░░  ░░░░░░  ░░░░░░░░░░░░░░██
               ██░░  ░░░░░░  ░░░░░░  ░░░░░░  ░░██
               ██████████████████████████████████
                   ██▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒██            -- SDKs
               ██████████████████████████████████
               ██▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒██        -- APIs
               ██▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒██
                 ██████████████████████████████          -- DATABASES
               ██                              ██        -- PROTOCOLS 
                  ████████  ████░░████  ████  ██
               ██░░██░░░░██░░░░██░░░░██░░░░██░░██        -- BACKEND
               ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██
                 ██████████████████████████████          -- ENGINES/PLATFORMS
               ██████████████████████████████████