Smart Crossfade on Custom Function Still Duplicates Metadata

[BusterNeece]

Liquidsoap 2.1.0 via Debian package on GitHub releases

Describe the bug Because we want crossfading to have custom rules when fading to the live broadcast, we can't just use the standard crossfade function that now has the code in it to avoid duplicate metadata. Therefore, it appears we're still seeing duplicate metadata in circumstances where cross.smart reverts to its default function, which happens more often in shorter tracks with jingles.

Here's our custom crossfade code:

def live_aware_crossfade(old, new) =
    if !to_live then
        # If going to the live show, play a simple sequence
        sequence([fade.out(old.source),fade.in(new.source)])
    else
        # Otherwise, use the smart transition
        cross.smart(old, new, fade_in=2.00, fade_out=2.00)
    end
end

radio = cross(minimum=0., duration=3.00, live_aware_crossfade, radio)

Because we now depend on LS's on_metadata handler for our own now-playing tracks, this means that we often see metadata sent to us in this order:

• Track A
• Jingle
• Track A
• Track B

Since it's only happening with shorter tracks (like jingles), it's pretty clear that this is caused by the smart crossfade failing over to the default sequence function, which is then prone to duplicating metadata, but for which we don't have the metadata-duplication-prevention code that was added in newer LS versions because we aren't using crossfade directly.

There is a workaround posted to this in a previous thread that involves passing cross.smart a custom function that uses add instead of sequence, but when we used that, it resulted in some unusually long crossfade times as written. It's possible that a small modification to that would work well for us, but I'd like the team's advice here before proceeding.

[toots]

Thanks for reporting.

This could be caused by some residual metadata on the leaving source in the sequence. Have you tried to drop all metadata from this source? Something like this:

def live_aware_crossfade(old, new) =
    if !to_live then
        # If going to the live show, play a simple sequence
        sequence([drop_metadata(fade.out(old.source)),fade.in(new.source)])
    else
        # Otherwise, use the smart transition
        cross.smart(old, new, fade_in=2.00, fade_out=2.00)
    end
end

radio = cross(minimum=0., duration=3.00, live_aware_crossfade, radio)

[BusterNeece]

@toots I can certainly try it, but this is applying to streams that don't have live broadcasts, so that to_live branch shouldn't ever be executing at all. It's definitely using the cross.smart transition, and I can confirm that in the specific case of using the smart crossfade, it does this, but with other crossfades and with crossfading disabled, it doesn't.

[BusterNeece]

Following up on this, we've updated to the latest stable version and it appears the issue persists exactly as described. Mitigations include disabling smart crossfade or disabling crossfade altogether.

[toots]

Thanks @BusterNeece. How long are the jingles? Are they under the crossfade duration of 3s?

[BusterNeece]

@toots Not sure that they're under the crossfade duration itself, but they're short enough that the "smart crossfade" doesn't have enough track data to analyze. That seems to be the defining characteristic of the bug.

[toots]

We are about to rewrite the whole metadata layer for the next major release so I would be enclined to too at a work around.

Are you able to add custom liquidsoap operators? You could try to drop this at the beginning of your script:

# Simple transition for crossfade
# @category Source / Track Processing
# @param ~fade_in  Fade-in duration, if any.
# @param ~fade_out Fade-out duration, if any.
# @param a Ending track
# @param b Starting track
def cross.simple(~fade_in=3.,~fade_out=3.,a,b)
  def fade.out(s) = fade.out(type="sin",duration=fade_out,s) end
  def fade.in(s)  = fade.in(type="sin",duration=fade_in,s) end
  add = fun (a,b) -> add(normalize=false,[b, a])

  add(drop_metadata(fade.out(a)),fade.in(b))
end

# Smart transition for crossfade
# @category Source / Track Processing
# @param ~log Default logger
# @param ~fade_in  Fade-in duration, if any.
# @param ~fade_out Fade-out duration, if any.
# @param ~high     Value, in dB, for loud sound level.
# @param ~medium   Value, in dB, for medium sound level.
# @param ~margin   Margin to detect sources that have too different sound level for crossing.
# @param ~default Smart crossfade: transition used when no rule applies (default: sequence).
# @param a Ending track
# @param b Starting track
def cross.smart(~log=fun(x)->log(label="cross.smart",x),
                ~fade_in=3.,~fade_out=3.,
                ~default=(fun (a,b) -> (sequence([a, b]):source)),
                ~high=-15., ~medium=-32., ~margin=4.,
                a, b)
  def fade.out(s) = fade.out(type="sin",duration=fade_out,s) end
  def fade.in(s)  = fade.in(type="sin",duration=fade_in,s) end
  add = fun (a,b) -> add(normalize=false,[b, a])

  # This is for the type system..
  ignore(a.metadata["foo"])
  ignore(b.metadata["foo"])

  if
    # If A and B are not too loud and close, fully cross-fade them.
    a.db_level <= medium and b.db_level <= medium and abs(a.db_level - b.db_level) <= margin
    then
      log("Old <= medium, new <= medium and |old-new| <= margin.")
      log("Old and new source are not too loud and close.")
      log("Transition: crossed, fade-in, fade-out.")
      add(drop_metadata(fade.out(a.source)),fade.in(b.source))

  elsif
    # If B is significantly louder than A, only fade-out A.
    # We don't want to fade almost silent things, ask for >medium.
    b.db_level >= a.db_level + margin and a.db_level >= medium and b.db_level <= high
  then
    log("new >= old + margin, old >= medium and new <= high.")
    log("New source is significantly louder than old one.")
    log("Transition: crossed, fade-out.")
    add(drop_metadata(fade.out(a.source)),b.source)

  elsif
    # Opposite as the previous one.
    a.db_level >= b.db_level + margin and b.db_level >= medium and a.db_level <= high
    then
    log("old >= new + margin, new >= medium and old <= high")
    log("Old source is significantly louder than new one.")
    log("Transition: crossed, fade-in.")
    add(drop_metadata(a.source),fade.in(b.source))

  elsif
    # Do not fade if it's already very low.
    b.db_level >= a.db_level + margin and a.db_level <= medium and b.db_level <= high
  then
    log("new >= old + margin, old <= medium and new <= high.")
    log("Do not fade if it's already very low.")
    log("Transition: crossed, no fade.")
    add(drop_metadata(a.source),b.source)

  # What to do with a loud end and a quiet beginning ?
  # A good idea is to use a jingle to separate the two tracks,
  # but that's another story.

  else
    # Otherwise, A and B are just too loud to overlap nicely, or the
    # difference between them is too large and overlapping would completely
    # mask one of them.
    log("No transition: using default.")
    default(drop_metadata(a.source), b.source)
  end
end

# Crossfade between tracks, taking the respective volume levels into account in
# the choice of the transition.
# @category Source / Track Processing
# @param ~id           Force the value of the source ID.
# @param ~duration     Duration (in seconds) of buffered data from each track \
#                      that is used to compute the transition between tracks.
# @param ~override_duration \
#                      Metadata field which, if present and containing a \
#                      float, overrides the 'duration' parameter for current \
#                      track.
# @param ~fade_in      Fade-in duration, if any.
# @param ~fade_out     Fade-out duration, if any.
# @param ~width        Width of the volume analysis window.
# @param ~conservative Always prepare for a premature end-of-track.
# @param ~minimum      Minimum duration (in sec.) for a cross: \
#                      If the track ends without any warning (e.g. in case of skip) \
#                      there may not be enough data for a decent composition. \
#                      Set to 0. to avoid having transitions after skips, \
#                      or more to avoid transitions on short tracks. \
#                      With a negative default, transitions always occur.
# @param ~default      Smart crossfade: transition used when no rule applies \
#                      (default: sequence).
# @param ~smart        Enable smart crossfading
# @param ~high         Smart crossfade: value, in dB, for loud sound level.
# @param ~medium       Smart crossfade: value, in dB, for medium sound level.
# @param ~margin       Smart crossfade: margin to detect sources that have too different \
#                      sound level for crossing.
# @param ~deduplicate  Crossfade transitions can generate duplicate metadata. When `true`, the operator \
#                      removes duplicate metadata from the returned source.
# @param s             The input source.
def crossfade(~id=null(), ~duration=5.,~override_duration="liq_cross_duration",
              ~fade_in=3.,~fade_out=3.,~smart=false,
              ~default=(fun (a,b) -> (sequence([a, b]):source)),
              ~high=-15., ~medium=-32., ~margin=4., ~deduplicate=true,
              ~minimum=(-1.),~width=2.,~conservative=true,s)
  id = string.id.default(default="crossfade", id)
  def log(~level=3,x) = log(label=id,level=level,x) end

  def simple_transition(a,b)
    list.iter(fun(x)-> log(level=4,"Before: #{x}"), metadata.cover.remove(a.metadata))
    list.iter(fun(x)-> log(level=4,"After : #{x}"), metadata.cover.remove(b.metadata))

    log("Simple transition: crossed, fade-in, fade-out.")
    cross.simple(fade_in=fade_in, fade_out=fade_out, a.source, b.source)
  end

  def smart_transition(a,b)
    list.iter(fun(x)-> log(level=4,"Before: #{x}"), metadata.cover.remove(a.metadata))
    list.iter(fun(x)-> log(level=4,"After : #{x}"), metadata.cover.remove(b.metadata))

    cross.smart(log=log, fade_in=fade_in, fade_out=fade_out, default=default,
                high=high, medium=medium, margin=margin, a, b)
  end

  transition =
    if smart then
      smart_transition
    else
      simple_transition
    end

  let (cross_id, deduplicate_id) = deduplicate ? (null(), null(id)) : (null(id), null())

  s = cross(id=cross_id, width=width, duration=duration, override_duration=override_duration,
            conservative=conservative, minimum=minimum, transition, s)

  if deduplicate then
    metadata.deduplicate(id=deduplicate_id, s)
  else
    s
  end
end

[toots]

Hi @BusterNeece. I was debugging the upcoming multi tracks branch and found more possible cause for this issue. You might want to checkout the stable build coming up here: https://github.com/savonet/liquidsoap/actions/runs/3684757153

[toots]

Hmm this one in fact: https://github.com/savonet/liquidsoap/actions/runs/3684757153 or whatever ends up passing the rolling-release-v2.1.x build 😅

[BusterNeece]

@toots I've updated our Rolling Release branch to use your Rolling Release branch, we'll see if that helps with things. Thank you!