magenta
Feature Request: AI-Powered Audio Analysis Tool (Stem Splitting to Formal Notation)
Labels: Feature Request, Music, AI/ML
Problem Statement Current music technology, even with advanced AI, often creates a divide between acoustic processing (e.g., stem splitting) and formal music theory analysis. Musicians must manually transcribe and analyze isolated audio stems to derive harmonic data, which is inefficient.
Proposal: The Magenta Analyzer Utility We propose a feature or application built on Magenta's ML models (e.g., MT3 or DDSP) that provides a seamless, developer-friendly workflow from raw audio to formalized music theory data.
Key Functionality:
- Audio Ingestion & Stem Splitting: Accept an audio file (MP3/WAV), and use a state-of-the-art Stem Separation model to isolate individual components (specifically, Bass and Keys/Harmonic Stems are a priority).
- Targeted Analysis: Allow the user (or system) to select a single isolated stem for automated pitch and harmony analysis.
- Dual Notation Output (The Core Feature): The system should analyze the selected stem and output two synchronized representations: Formal Theory Notation (Roman Numerals): Display the underlying harmonic progression using standard Roman Numeral Analysis (e.g., $i - iv - V^7 - I$ in C minor) to facilitate quick structural and academic understanding. Performance Notation (Arabic Numerals/MIDI): Provide the note sequence using the Arabic Numeral (0-11) pitch-class set notation or MIDI note numbers for ease of live performance planning, keyboard key assignment, or DAW sequencing.
- Integration & Export: The analysis results (Roman numerals, note sequences, and timestamps) should be easily exportable as a structured format (e.g., JSON or a Google Sheet).
Value Proposition for Google/Magenta This feature bridges AI audio separation (acoustic) with formal music theory (academic/practical), positioning Magenta as a leader in comprehensive, smart tools for musicians and educators. It is an ideal project to showcase the power of Google's AI models in practical, creative applications.
