ultimatevocalremover_api
ultimatevocalremover_api copied to clipboard
API for a Vocal Remover that uses Deep Neural Networks.
Ultimate Vocal Remover API v0.1
This is a an API for ultimate vocal removing. It is designed to be expandable with new models/algorethems while maintaining a simple interface. Colab demo
Install
If you intend to edit the code
git clone https://github.com/NextAudioGen/ultimatevocalremover_api.git
cd ultimatevocalremover_api
pip install .
Usage
import uvr
from uvr import models
from uvr.utils.get_models import download_all_models
import torch
import audiofile
import json
models_json = json.load(open("/content/ultimatevocalremover_api/src/models_dir/models.json", "r"))
download_all_models(models_json)
name = {name_of_your_audio}
device = "cuda"
demucs = models.Demucs(name="hdemucs_mmi", other_metadata={"segment":2, "split":True}, device=device, logger=None)
# Separating an audio file
res = demucs(name)
seperted_audio = res["separated"]
vocals = seperted_audio["vocals"]
base = seperted_audio["bass"]
drums = seperted_audio["drums"]
other = seperted_audio["other"]
Archetecture:
Ultimate Vocal Remover API
├── src
│ ├── audiotools.py
│ ├── models.py
│ ├── ensembles.py
│ ├── pipelines.py
│ ├── utils/
│ ├── audio_tools/
│ └── models_dir
│ ├── Each implementation of a model is added here as a single directory.
│ └── models.json (this is used to download the models)
├── docs
│ ├── models/
│ │ └── Here goes all models docs each in a single directory.
│ ├── ensembles/
│ │ └── Here goes all ensembles docs each in a single directory.
│ ├── pipelines/
│ │ └── Here goes all pipelines docs each in a single directory.
│ ├── audio_tools/
│ └── utils/
└── tests/
├── test_models.py
├── test_ensembles.py
├── test_pipelines.py
├── test_audiotools.py
└── utils/
audiotools.py: Interface for all audio tools
models.py: Interface for all models following a consistent interface
utils/ Here goes read and write utils for audio, models...etc. \
All models, pipelines and ensembles follow this interface:
class BaseModel:
def __init__(self, name:str, architecture:str, other_metadata:dict, device=None, logger=None)
def __call__(self, audio:Union[npt.NDArray, str], sampling_rate:int=None, **kwargs)->dict
# @singledispatch
def predict(self, audio:npt.NDArray, sampling_rate:int, **kwargs)->dict
def predict_path(self, audio:str, **kwargs)->dict
def separate(self, audio:npt.NDArray, sampling_rate:int=None)->dict
def __repr__(self)
def to(self, device:str)
def update_metadata(self, metadata:dict)
@staticmethod
def list_models()->list
Contribution
If you like this, leave a star, fork it, and definitely you are welcomed to buy me a coffee.
Also, please open issues, make pull requests but remember to follow the structure and interfaces. Moreover, we are trying to build automated testing, we are aware that the current tests are so naive but we are working on it. So please make sure to add some tests to your new code as well.
Refrences
code
Code and weights from these sources used in developing this library:
- MDX-Net This is the original MDX architecture implementation.
- MDXC and demucs This repo has a clever ensumbling methods for MDX, Demucs 3, and Demucs 4. Moreover they have the wieghts for their finetuned MDX open (available under MDXC implementation here).
- Demucs This is the original implementation of the model.
- ultimatevocalremovergui This is one of the best vocal removers. A lot of ideas in this repo were borrowed from here.
- weights Most of the models right now are comming from this repo.
Papers
- Benchmarks and leaderboards for sound demixing tasks
- MULTI-SCALE MULTI-BAND DENSENETS FOR AUDIO SOURCE SEPARATION
- HYBRID TRANSFORMERS FOR MUSIC SOURCE SEPARATION
- KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixing