pyannote-core icon indicating copy to clipboard operation
pyannote-core copied to clipboard

plot_annotation by labels, instead of optimally separating segments

Open janaab11 opened this issue 4 years ago • 2 comments

The default implementation seems to separate overlapping segments into different lines (vertically), by parsing the start and end times of all segments, linearly from left to right. [source]

While this is pretty useful visually, there is another way to handle overlaps: plot differently-labelled segments on separate lines. In particular, this is useful if the different labels represent different audio environments and you want to visualize differences in your model's performance across environments/labels.

The actual implementation of this is pretty simple, and adds only three new lines of code:

  def plot_annotation(self, annotation: Annotation, ax=None, time=True, legend=True, separate_by="optimal"):

      if not self.crop:
          self.crop = annotation.get_timeline(copy=False).extent()

      cropped = annotation.crop(self.crop, mode='intersection')
      labels = cropped.labels()
      labels_dict = {label:i for i, label in enumerate(labels)}
      segments = [s for s, _ in cropped.itertracks()]

      ax = self.setup(ax=ax, time=time)

      for (segment, track, label), y in zip(
              cropped.itertracks(yield_label=True),
              self.get_y(segments)):
          if separate_by == "labels":
              y = 1. - 1. / (len(labels) + 1) * (1 + labels_dict.get(label))
          self.draw_segment(ax, segment, y, label=label)

Let me know if this works and i can make a PR!

An example of this implementation at work. The model clearly underperforms in the SYS label, by comparison: human-vad chonker-vad

janaab11 avatar Jun 09 '21 13:06 janaab11

The visualisation part of pyannote.core definitely needs some love! I'd likely merge a PR adding this kind of feature. Thanks!

Not sure how to name the option though. What about arrangement="pack" (current behavior) vs arrangement="stack" (what you propose)?

hbredin avatar Jun 14 '21 09:06 hbredin

The names you suggest sound apt for this option. I will put through a PR with these changes, this weekend - sorry about the delay.

janaab11 avatar Jun 30 '21 17:06 janaab11