Reality-Synthesizer icon indicating copy to clipboard operation
Reality-Synthesizer copied to clipboard

verified publisher icon robomex
→

Metadata

iOS 15.4 AVFoundation LiDAR sensor + MIDI notes = depth-based augmented reality music visualization as the music is being played

trafficstars

Reality Synthesizer: Visualize MIDI Notes in Augmented Reality

iOS 15.4 opened up the LiDAR sensor to AVFoundation. In this demo MIDI notes played on a paired synthesizer are interpreted by Metal shaders to produce augmented reality visualizations. This demo builds on parts of Capturing Depth Using the LiDAR Camera, AudioKit, and Linnstrument Helper.

The demo with sound can be viewed at this tweet.

Reality Synthesizer Demo Gif

← Metadata

21
Stars
1
Forks
Watchers

Owner

verified publisher icon robomex

Metadata

iOS 15.4 AVFoundation LiDAR sensor + MIDI notes = depth-based augmented reality music visualization as the music is being played

Back