asteroid
asteroid copied to clipboard
asteroid-team
Chime4
About this PR
This PR introduces a recipe for evaluation purposes only, on real noisy data. It uses the data from the CHiME 4 challenge (an adaptation from the CHiME 3 challenge).
Design
I am unsure about the design of the dataset and the recipe :
- The dataset mimics the behaviour of other datasets by returning mixture and sources(here fake sources that are actually a copy of the mixture)
- The recipe only contains scripts for the evaluation it might be worth to make it more general by avoiding to limit its application to only one model
I'm undecided about this. On the one hand, having data prep and eval script for Chime4 is cool, and it's an example for other models but we could probably make more modular than this.
But to make it more modular, we should attach the data prep to the dataloader (needs refactoring of the data part of Asteroid, which we want to do), and we should enable evaluating any model on any dataset (also needs refactoring).
I'd vote in favour of merging it, because it diversifies the examples that we have (both for the users and things in mind when we refactor).
@jonashaag @popcornell
Yes, I'd say merge it, as it's not introducing any new way of doing things, so it doesn't add much work to our refactoring plate.
Did you forget to rename the "chime3" variables and CLI flag or is that on purpose?
Yes, I'd say merge it, as it's not introducing any new way of doing things, so it doesn't add much work to our refactoring plate.
Did you forget to rename the "chime3" variables and CLI flag or is that on purpose?
That's on purpose because CHiME 4 uses data from CHiME 3 when you download the data you actually get a directory named CHiME3. But maybe it's confusing.
I guess it will be clear once people have the data from CHiME3 downloaded.