audtorch icon indicating copy to clipboard operation
audtorch copied to clipboard

WIP: Add Musdb18 data set

Open hagenw opened this issue 5 years ago • 6 comments

Summary

Add musdb18 which is a data set for musical source separation.

Proposed Changes

  • add datasets/musdb18.py
  • add musdb as dependency to setup.py
  • add Musdb18 to documentation

Discussion

  1. Implemenation: the data set cannot be based on our AudioDataset as the data set comes in a special format and requires the external package musdb to read the files. Alternatively, there is the option to first convert it to WAV and then use our normal approach. NOTE: in order to work musdb needs ffmpeg installed on your system.

  2. Automatic download: is not yet included as the data set cannot be freely downloaded, but you have to ask for permission first. There is a short version of the data set (7s excerpts) that can be automatically downloaded. So we might think about including those.

  3. percentage_silence: this idea is from the SpeechNoiseMix data set. It should help to force your trained model to return silence for parts of the signal where the target source is not active (e.g. no speaker talking for SpeechNoiseMix or no singing vocal in the case of this data set).

hagenw avatar Jul 11 '19 10:07 hagenw

Thanks for adding this corpus. Could you explain the idea behind percentage_silence? I didn't fully get what happens to signal and target.

phtephanx avatar Jul 11 '19 11:07 phtephanx

Yeah, I forgot to mention that in the discussion, I added it there now.

hagenw avatar Jul 11 '19 12:07 hagenw

I'm interested in reviving this, if you are too @hagenw. It would require some work to rebase, but I think the code is already good and we could easily add.

Is there some reason you've marked it as a WIP?

ATriantafyllopoulos avatar Nov 14 '19 08:11 ATriantafyllopoulos

I marked it as WIP as it was only a first try. Then I stopped working on it, as I thought we would internally convert it to our format and this is no longer needed.

hagenw avatar Nov 14 '19 08:11 hagenw

True, but we could think of adding it to audtorch as it is now. I think it's working and does what it should do, right? So since the code is already there, the community might still benefit from it. Or do you think it needs to be reworked?

ATriantafyllopoulos avatar Nov 14 '19 10:11 ATriantafyllopoulos

No, you are right, we should integrate it into audtorch. I will start with doing the rebase first.

What is still missing is automatic download of the data set, as the community expects this as well, but I think we could start without as you are only allowed to download the 7s versions of the files anyway.

hagenw avatar Nov 14 '19 10:11 hagenw