MusicBoundariesCNN
MusicBoundariesCNN copied to clipboard
Code of the paper "Music Boundary Detection using Convolutional Neural Networks: A comparative analysis of combined input features" in Pytorch
![](https://www.unizar.es/sites/default/files/identidadCorporativa/imagen/logoUZ.png)
MUSIC BOUNDARIES DETECTION (STRUCTURE SEGMENTATION)
Code repository for the paper Music Boundary Detection using Convolutional Neural Networks: A comparative analysis of combined input features. See arXiv preprint Currently being reviewed in the International Journal of Interactive Multimedia and Artifcial Intelligence Journal.
Introduction
Music Structure Segmentation is a research part in Music Information Retrieval (MIR). Since 2009, MIREX's campaigns have been tested this algorithms which are composed by unsupervised and supervised neural networks methods. This methods take as inputs audio features such as MFCCs, chroma vectors or spectrograms, and the well-known self-similarity (lag) matrices SSM or SSLM.
Check SelfSimilarityMatrices repository and see the notebooks to follow the procedure of the inputs (SSLMs) calculation step by step.
![](https://github.com/carlosholivan/MusicBoundariesCNN/raw/master/results/song1076_mls.png)
TODOs
- [ ] Fix Training script
- [ ] Fix Evaluation script
- [ ] Make a test script where an audio file is taken and the prediction is given as the output
Package Structure
boundariesdetectioncnn/
module containing VAE architecture, training and data uitilities.
boundariesdetectioncnn/models
CNN models.
boundariesdetectioncnn/data
data handling tools.
boundariesdetectioncnn/train
model training tools.
boundariesdetectioncnn/evaluation
model evaluation tools.
notebooks/
tutorial notebooks.
tests/
unit tests.
Installation
cd .path/to/timbre-vae
python setup.py install
Prerequisites
Python 3.5 or later. In Ubuntu, Mint and Debian Python 3 can be installed like this:
sudo apt-get install python3 python3-pip
sudo pip install librosa
If you use conda/Anaconda environments, librosa can be installed from the conda-forge channel:
conda install -c conda-forge librosa
Databases
Here is a list of the databases that are used in Music Structural Analysis. This model has been trained, evaluated and tested with SALAMI 2.0 dataset.
RCW Database
RCW Goto Annotations: http://staff.aist.go.jp/m.goto/RWC-MDB/AIST-Annotation
RCW Quaero Project Annotations (MIREX10): http://musicdata.gforge.inria.fr/
Beatles Database
Beatles-TUT Annotations: http://www.cs.tut.fi/sgn/arg/paulus/beatles_sections_TUT.zip
Isophonic Beatles or Beatles-ISO Annotations: http://isophonics.net/content/reference-annotations
SALAMI 2.0 Database
2.0 version: https://ddmal.music.mcgill.ca/research/SALAMI/
Webs of Interest
References
[1] | Cohen-Hadria, A., & Peeters, G. (2017, June). Music structure boundaries estimation using multiple self-similarity matrices as input depth of convolutional neural networks. In Audio Engineering Society Conference: 2017 AES International Conference on Semantic Audio. Audio Engineering Society. |
[2] | Grill, T., & Schlüter, J. (2015, October). Music Boundary Detection Using Neural Networks on Combined Features and Two-Level Annotations. In ISMIR (pp. 531-537). |
[3] | Grill, T., & Schluter, J. (2015, August). Music boundary detection using neural networks on spectrograms and self-similarity lag matrices. In 2015 23rd European Signal Processing Conference (EUSIPCO) (pp. 1296-1300). IEEE. |
[4] | Serra, J., Müller, M., Grosche, P., & Arcos, J. L. (2012, July). Unsupervised detection of music boundaries by time series structure features. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 26, No. 1). |
Authors
- Carlos Hernández - [email protected]
- David Díaz-Guerra - [email protected]
- José Ramón Beltrán - [email protected]