mir_eval
mir_eval copied to clipboard
Numerical stability issues in floating point comparisons (chord directional hamming)
I ran into a funny bug just now, and wanted to get some opinions on whether this should be considered an mir_eval problem or not.
This check in the directional hamming distance function looks for disjoint time intervals by comparing the end of one interval to the start of the next.
This is fine in principle, but I have a few (reference) files which are stored in JSON (jams) format, and for whatever reason, deserializing them messes up the floating point precision just right so that it triggers an error. For example:
In [4]: J = jams.load('/home/bmcfee/data/crema/chords/TRAIIEF149E3861F6C.jams')
In [5]: ints, labs = J.annotations[0].to_interval_values()
In [6]: ints[1, 1]
Out[6]: 2.126712
In [7]: ints[2, 0]
Out[7]: 2.1267119990000003
On the one hand, this is clearly a data error, and it's reasonable for mir_eval to fail here.
On the other hand, this is not a serious data error, and it's one that we can expect to pop up any time we're dealing with numerical data through text serialization, which is basically every use case of mir_eval.
Given that, what do people think of rounding the time intervals (say to the millisecond level) prior to checking? That is, just add lines to the effect of
reference_intervals = np.around(reference_intervals, decimals=3)
before the overlap check. For my money, I see no downside to this, but I'd like to get some opinions before opening a PR.