sigsep-mus-eval
sigsep-mus-eval copied to clipboard
Allow silent targets and estimates
Hello,
I'm proposing this PR following the discussions of #94. With this implementation, mus-eval raises a warning instead of an error if one of the sources or estimates is completely silent and compute_permutation
is False
(if it is True
it raises the same error it used to).
Some important notes:
- It handles silent sources or estimates in a way equivalent to how the library already handled silent frames: the metric is set to
NaN
or-Inf
in that song and then the song is excluded when computing the average metric of the dataset. - This does not modify any of the results that could be computed before, it just allows getting results for inputs that raised errors.
I think this is important for dealing with datasets more complex than MusDB18, where a wider variety of instruments could be available in the whole dataset but not all the instruments might be present in every song (this situation could be quite common, for example, in classical music).
Best, David
Sorry for the bug in the previous version. Now the only reason the tests are not passed is that test_silent_input
is obviously not raising a ValueError
. I'm not sure if you would prefer to modify or just remove that test.
@DavidDiazGuerra maybe modify the tests
Done!