liquidsoap
liquidsoap copied to clipboard
Published
20 hours ago •
savonet
savonet
Unknown Track Title while using multiple fallbacks with input.harbor and crossfade
Describe the bug
This is working on 2.0.6 but it looks like this issue has returned in 2.1.4 with the first track showing Unknown on Icecast with input.harbor and crossfade Looks to have been fixed previously: https://github.com/savonet/liquidsoap/issues/2138
Output from liquidsoap
2023/08/24 19:04:59 [/zonnebloem:3] New metadata chunk ? -- De Zonnebloem Live uit Beilen.
2023/08/24 19:05:01 [switch_2:3] Switch to LiveBroadcast with transition.
2023/08/24 19:05:01 [source:4] Source replay_metadata_575 gets down.
2023/08/24 19:05:01 [source:4] Source replay_metadata_579 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/08/24 19:05:01 [replay_metadata_579:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/08/24 19:05:01 [Standard_Request:4] Finished with "/usr/local/mediacp/content/user_25/music/muziek/Anja Johnny - Had Ik Jou Maar Nooit Gezien.mp3".
2023/08/24 19:05:01 [LiveBroadcast:3] Switch to /zonnebloem.
2023/08/24 19:05:01 [source:4] Source replay_metadata_580 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/08/24 19:05:01 [replay_metadata_580:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/08/24 19:05:01 [lang:3] [on_metadata_update] empty metadata
To Reproduce liquidsoap.liq
settings.init.daemon.set(true);
settings.init.daemon.pidfile.set(true);
settings.init.daemon.pidfile.path.set("/usr/local/mediacp/content/user_1/icecast_kh/8008_90/liquidsoap.pid");
settings.server.socket.set(true);
settings.server.socket.path.set("/usr/local/mediacp/content/user_1/icecast_kh/8008_90//liquidsoap.sock")
settings.log.file.set(true);
settings.log.file.path.set("/usr/local/mediacp/content/user_1/icecast_kh/8008_90/logs/liquidsoap.log");
settings.log.stdout.set(true);
settings.log.level.set(4);
settings.tag.encodings.set(["UTF-8","ISO-8859-1"]);
#settings.encoder.metadata.export.set(["artist","title","album","song"])
settings.scheduler.non_blocking_queues.set(4)
settings.scheduler.generic_queues.set(4)
settings.scheduler.fast_queues.set(4)
settings.request.metadata_decoders.set(["TAGLIB", "FLAC", "OGG"])
settings.request.leak_warning.set(100)
#settings.request.grace_time.set(600.)
settings.harbor.bind_addrs.set(["0.0.0.0"]);
settings.harbor.timeout.set(300.)
settings.harbor.verbose.set(true)
queueDebugging = false
recordLiveBroadcasts = false
service_id = 90
harbor_buffer = 12
harbor_max_buffer = 20
crossfade_duration = 5
harbor_clear_queue = 60 # Don't return to original track on harbor disconnection if this has passed in seconds
dj = ref("");
dj_username = ref("");
previousMeta = ref([]);
startRecording = ref (fun () -> ())
# Track live transition for crossfades.
to_live = ref(false)
ignore(to_live)
def r(type) =
requestUrl = "https://127.0.0.1:2021/controller/AudioPlayout/90/#{type}";
log("#### [Request: #{type}] Requesting next track from #{requestUrl}");
ret = process.read("curl -k " ^ requestUrl);
log("#### [Request: #{type}] Got Answer: " ^ ret);
if ret == "" or string.match(pattern="Error", ret) or string.match(pattern="Exception", ret) or string.match(pattern="no-track", ret) then
[]
else
r = request.create(ret)
if request.resolve(r) then
[r]
else
[]
end
end
end
def fetchAndRetry(s) =
thread.run.recurrent(fast=false, delay=0., fun() -> begin
log("Running s.fetch()");
if s.fetch() then
-1.
else
2. # Check every 2 seconds until ready
end
end);
end
def q(remaining,s) =
log("Queueing #{s.id()} with #{remaining} seconds remaining");
s.fetch();
()
end
def harborOnConnect(headers) =
log("### Harbor connected");
if recordLiveBroadcasts then
log("#### Broadcast recording starting");
srf = !startRecording
srf()
()
end
end
def harborOnDisconnect() =
log("### Harbor disconnected");
dj_username := "";
dj := ""
end
def harborAuth(auth) =
log("#### Harbor DJ Auth is: " ^ auth.user ^ ":" ^ auth.password);
log("#### /usr/local/mediacp/php/bin/php /usr/local/mediacp/htdocs/system/misc/scripts/radio_auth.php --server-type=icecast_kh --server-id=90 --username=" ^ process.quote(auth.user) ^ " --password=" ^ process.quote(auth.password));
ret = process.read.lines("/usr/local/mediacp/php/bin/php /usr/local/mediacp/htdocs/system/misc/scripts/radio_auth.php --server-type=icecast_kh --server-id=90 --username=" ^ process.quote(auth.user) ^ " --password=" ^ process.quote(auth.password))
if list.hd(default="",ret) == "authenticated" then
dj_username := auth.user
dj := list.nth(
default=
if auth.user == "source" then
list.hd(default="",string.split(separator=":",auth.password))
else
auth.user
end
,ret,1)
log("#### Harbor authentication success");
log("#### Harbor connected as " ^ !dj);
true
else
log("#### Harbor authentication failed");
false
end
end
def metadata_updated(m) =
if list.length(m) == 0 then
log("[on_metadata_update] empty metadata")
elsif string_of(!previousMeta) == string_of(m) then
log("[on_metadata_update] no change since last track, skipping feedback")
elsif m["title"] == "" then
log("[on_metadata_update] title is empty, skipping feedback")
else
previousMeta := m # Update previousMeta
thread.run(fast=false, delay=0., fun() -> begin
log("###### Metadata Feedback Update");
if (m["track_id"] != "") then
log("track_id:" ^ m["track_id"]);
log("Title: " ^ m["title"])
log("Artist: " ^ m["artist"])
log("Album: " ^ m["album"])
log(string.quote(m["title"]));
url = '/usr/local/mediacp/php/bin/php /usr/local/mediacp/htdocs/system/misc/scripts/feedback.php' ^
" server_id=90" ^
' track_id=' ^ string.quote(m["track_id"]) ^
' playlistable_id=' ^ string.quote(m["playlistable_id"]) ^
' playlistable_type=' ^ string.quote(m["playlistable_type"]) ^
' playlist_position=' ^ string.quote(m["playlist_position"]) ^
#' dj_id=' ^ string.quote(m["track_id"]) ^
' title=' ^ string.quote(m["title"]) ^
' artist=' ^ string.quote(m["artist"]);
log(url);
ret = list.hd(process.read.lines(url),default="")
log("MediaCP Feedback Response: #{ret}")
else
log("Filename: " ^ m["filename"])
log("Title: " ^ m["title"])
log("Artist: " ^ m["artist"])
log("Album: " ^ m["album"])
log("Dj: " ^ !dj)
url = '/usr/local/mediacp/php/bin/php /usr/local/mediacp/htdocs/system/misc/scripts/feedback.php' ^
" server_id=90" ^
' dj=' ^ string.quote(!dj) ^
' text=' ^ string.quote(m["title"]);
log(url);
ret = list.hd(process.read.lines(url),default="")
log("MediaCP Feedback Response: #{ret}")
end
()
end)
end
()
end
def processReplayGain(s) =
if false then
log("Triggering enable_replaygain_metadata()");
enable_replaygain_metadata()
replaygain(s)
else
s
end
end
def live_aware_crossfade(old, new) =
log("#### live_aware_crossfade");
if !to_live then
# If going to the live show, play a simple sequence
log("#### Sequence Crossfade to Live DJ")
sequence([fade.out(old.source),fade.in(new.source)])
elsif false then
log("#### Smart Crossfade between tracks - in=2.00 out=3.00");
cross.smart(old, new, fade_in=2.00, fade_out=3.00)
else
log("#### Simple Crossfade between tracks - in=2.00 out=3.00");
cross.simple(old.source, new.source, fade_in=2.00, fade_out=3.00)
end
end
def processCrossfade(s) =
if true then
log("#### Crossfade enabled - in=2.00 out=3.00");
cross(minimum=0., duration=float_of_int(crossfade_duration), live_aware_crossfade, s)
else
log("#### Crossfade disabled");
s
end
end
# Standard Priority Requests
standardRequest = request.dynamic.list(id="Standard_Request", prefetch=0, { r('standard') })
standardPlayout = source.on_end(
id="Standard Playout",
delay=10.0,
standardRequest,
fun(remaining, _) -> begin
if list.length(standardRequest.queue()) < 1 then
log("Queueing Standard_Request with #{remaining} seconds remaining");
fetchAndRetry(standardRequest)
()
end
()
end
);
# High Priority Requests - ping every 20 seconds
priorityRequest = request.dynamic.list(id="Priority Request", prefetch=0, { r('priority') })
priorityPlayout = source.on_end(
id="Priority Playout",
delay=10.0,
priorityRequest,
fun(remaining, _) -> begin
if list.length(priorityRequest.queue()) < 1 then
log("Queueing Priority_Request with #{remaining} seconds remaining");
if
# Attempt fetch
not priorityRequest.fetch()
# If no more priority requests, and no queued standard, then fetch fetch fetch
and list.length(standardRequest.queue()) == 0
and standardPlayout.remaining() == 0.
then
log("No more priority requests, fetching from standard queue instead");
fetchAndRetry(standardRequest)
()
end
()
end
()
end
);
thread.run(fast=false, delay=10., every=20., fun() -> begin
# Ping for new requests only if the priorityRequest source is not already active, since queueOnEndTrack would run if its active
if priorityRequest.is_ready() then () else priorityRequest.fetch(); () end
end)
# Add Skip to Playouts
playoutGroup = fallback(id="Playout Group", track_sensitive=false, [priorityPlayout, standardPlayout]);
server.register(
namespace="#{source.id(playoutGroup)}",
usage="skip",
description="Skip the current track.",
"skip",
fun(_) ->
begin
log("Skip Executed" ^ string_of(priorityRequest.is_ready()));
# Check if priority playing, attempt next request
if (priorityRequest.is_ready() and priorityRequest.fetch()) or standardRequest.fetch() then
thread.run(fast=false, delay=0.75,fun() -> begin source.skip(playoutGroup) end)
end
"Done!"
end
)
# Harbor connection
defaultDj = input.harbor(id="/", port=6818, buffer=float_of_int(harbor_buffer), max=float_of_int(harbor_max_buffer), icy=true, password="password", on_connect=harborOnConnect, on_disconnect=harborOnDisconnect, auth=harborAuth, "");
output.dummy(id="defaultDj", fallible=true, defaultDj);
liveBroadcast = defaultDj;
# Record live broadcasts
stopRecordingFunc = ref (fun () -> ())
def stopRecording() = f = !stopRecordingFunc; f();stopRecordingFunc := { () }; end
startRecording := fun() -> begin
log("Initialising recording");
stopRecording()
currentDj = !dj_username
outputRecording = output.file(
%mp3(samplerate=44100, stereo=true, bitrate=128, id3v2=true),
fallible=true,
#flush=true,
append=true,
reopen_on_metadata=false,
reopen_on_error=true,
on_stop = fun() -> begin
log("#### Inform MediaCP of new file recording");
# Allow plenty of time for file to finish writing
thread.run(fast=false, delay=5.,fun() -> begin
ret = process.read("/usr/local/mediacp/php/bin/php /usr/local/mediacp/htdocs/system/misc/scripts/notify.php server_id=90")
log("MediaCP Recording Notify Response: #{ret}")
end)
end,
"/usr/local/mediacp/content/user_1/music/recordings/%Y/%m/%Y-%m-%d-%H_%M_%S_#{currentDj}.mp3",
liveBroadcast
)
stopRecordingFunc := { outputRecording.shutdown() }
end
# Play blank if nothing available
radio = fallback(track_sensitive=false,[liveBroadcast, playoutGroup, blank(id="Initial Blank Startup")]) # Play blank, infaillible source if nothing else is available
radio.on_metadata(metadata_updated)
# Skip non-live track when live DJ goes live.
def check_live() =
if liveBroadcast.is_ready() then
if not !to_live then
to_live := true
playoutGroup.skip()
end
else
to_live := false
end
end
# Continuously check on live.
radio = source.on_frame(radio, check_live)
# ReplayGain (if enabled)
radio = processReplayGain(radio)
# Add Crossfading
radio = processCrossfade(radio);
radio = mksafe(radio)
output.icecast(
%fdkaac(
channels=2,
samplerate=44100,
bitrate=128,
afterburner=true,
aot="mpeg4_aac_lc",
transmux="adts",
sbr_mode=false
),
mount = "/stream",
host = "127.0.0.1",
port = 8008,
password = "password",
encoding = "UTF-8",
name = "",
description = "",
url = "",
genre = "",
public = true,
icy_metadata = "true",
radio
); output.icecast(
%mp3(
samplerate=11025,
bitrate=64,
id3v2=true,
stereo=true
),
mount = "/stream2",
host = "127.0.0.1",
port = 8008,
password = "passsword",
encoding = "UTF-8",
name = "",
description = "",
url = "",
genre = "",
public = false,
icy_metadata = "true",
radio
);
# Initialise first standard request once we're up and running
fetchAndRetry(standardRequest);
if queueDebugging then
thread.run(fast=true, delay=5., every=20., fun() -> begin
log(
"Standard[Queued: " ^ string_of(list.length(standardRequest.queue())) ^ " Remaining: " ^string_of(standardPlayout.remaining())^"] " ^
"Priority[Queued: " ^ string_of(list.length(priorityRequest.queue())) ^ " Remaining: " ^string_of(priorityPlayout.remaining())^"] " ^
"Harbor: " ^ string_of(liveBroadcast.is_ready())
)
() end)
end
# If harbor is connected for more than 30 seconds, clear the queue as to not return midway through the original trackd empty queue)
harbor_ping_count = ref(0)
harbor_ping_reached = ref(false)
thread.run(fast=false, delay=1.0, every=10., fun() -> begin
if not liveBroadcast.is_ready() then
harbor_ping_count := 0
harbor_ping_reached := false;
()
else
if !harbor_ping_count >= harbor_clear_queue and not !harbor_ping_reached then
log("Clear standard and priority queues so harbor does not return to mid track");
harbor_ping_reached := true;
source.skip(standardRequest);
source.skip(priorityRequest);
()
else
harbor_ping_count := !harbor_ping_count + 10; # Increment by 2, since thread.run is every 2 seconds
log("Harbor Ping Count: #{harbor_ping_count}");
()
end
end
end);
# If priority playlist is connected for more than 60 seconds, clear the standard queue as to not return midway through the original trackd empty queue)
priority_ping_count = ref(0)
priority_ping_reached = ref(false)
thread.run(fast=false, delay=1.0, every=10., fun() -> begin
if not priorityPlayout.is_ready() then
priority_ping_count := 0
priority_ping_reached := false;
()
else
if !priority_ping_count >= 60 and not !priority_ping_reached then
log("[priority_ping] Clear standard queue so priority does not return to middle of a standard request track");
priority_ping_reached := true;
source.skip(standardRequest);
()
else
priority_ping_count := !priority_ping_count + 10; # Increment by 2, since thread.run is every 2 seconds
log("Priority Ping Count: #{priority_ping_count}");
()
end
end
end);
Version details
- OS: Debian 11
- Version 2.1.4
Packages Installed
# Packages matching: installed
# Name # Installed # Synopsis
angstrom 0.15.0 Parser combinators built for speed and memory-
base-bigarray base
base-threads base
base-unix base
bigstringaf 0.9.1 Bigstring intrinsics and fast blits based on m
camomile 1.0.2 A Unicode library
conf-fdkaac 1 Virtual package relying on fdk-aac
conf-ffmpeg 1 Virtual package relying on FFmpeg
conf-lame 1 Virtual package relying on lame
conf-libcurl 2 Virtual package relying on a libcurl system in
conf-libflac 1 Virtual package relying on libFLAC
conf-libogg 1 Virtual package relying on libogg
conf-libpcre 1 Virtual package relying on a libpcre system in
conf-libtheora 1 Virtual package relying on libtheora
conf-libvorbis 1 Virtual package relying on libvorbis
conf-mad 1 Virtual package relying on mad
conf-pkg-config 3 Check if pkg-config is installed and create an
conf-taglib 1 Virtual package relying on taglib
conf-which 1 Virtual package relying on which
cry 0.6.7 OCaml client for the various icecast & shoutca
csexp 1.5.2 Parsing and printing of S-expressions in Canon
dtools 0.4.5 Library providing various helper functions to
dune 3.10.0 Fast, portable, and opinionated build system
dune-configurator 3.10.0 Helper library for gathering system configurat
duppy 0.9.2 Library providing monadic threads
faad 0.5.0 pinned to version 0.5.0
fdkaac 0.3.3 Fraunhofer FDK AAC Codec Library
ffmpeg 1.1.7 Bindings for the ffmpeg libraries
ffmpeg-av 1.1.7 Bindings for the ffmpeg libraries -- top-level
ffmpeg-avcodec 1.1.7 Bindings for the ffmpeg avcodec library
ffmpeg-avdevice 1.1.7 Bindings for the ffmpeg avdevice library
ffmpeg-avfilter 1.1.7 Bindings for the ffmpeg avfilter library
ffmpeg-avutil 1.1.7 Bindings for the ffmpeg avutil libraries
ffmpeg-swresample 1.1.7 Bindings for the ffmpeg swresample library
ffmpeg-swscale 1.1.7 Bindings for the ffmpeg swscale library
flac 0.3.1 Bindings to libflac
gen 1.1 Iterators for OCaml, both restartable and cons
lame 0.3.7 MP3 encoding library
liquidsoap 2.1.4 pinned to version 2.1.4
mad 0.5.2 Mad decoding library
menhir 20230608 An LR(1) parser generator
menhirLib 20230608 Runtime support library for parsers generated
menhirSdk 20230608 Compile-time library for auxiliary tools relat
mm 0.8.4 The mm library contains high-level APIs to cre
ocaml 4.12.1 The OCaml compiler (virtual package)
ocaml-base-compiler 4.12.1 Official release 4.12.1
ocaml-compiler-libs v0.12.4 OCaml compiler libraries repackaged
ocaml-config 2 OCaml Switch Configuration
ocaml-options-vanilla 1 Ensure that OCaml is compiled with no special
ocaml-syntax-shims 1.0.0 Backport new syntax to older OCaml versions
ocamlfind 1.9.6 A library manager for OCaml
ocurl 0.9.2 Bindings to libcurl
ogg 0.7.4 Bindings to libogg
pcre 7.5.0 Bindings to the Perl Compatibility Regular Exp
ppx_derivers 1.2.1 Shared [@@deriving] plugin registry
ppxlib 0.30.0 Standard infrastructure for ppx rewriters
result 1.5 Compatibility Result module
sedlex 3.2 An OCaml lexer generator for Unicode
seq base Compatibility package for OCaml's standard ite
sexplib0 v0.16.0 Library containing the definition of S-express
stdlib-shims 0.3.0 Backport some of the new stdlib features to ol
stringext 1.6.0 Extra string functions for OCaml
taglib 0.3.10 Bindings for the taglib library
theora 0.4.0 Bindings to libtheora
uri 4.2.0 An RFC3986 URI/URL parsing library
vorbis 0.8.0 Bindings to libvorbis
I’d say this is still in 2.3.0, if I’m not mistaken, and bugs users, show hosts, DJs and station owners alike—you simply can’t tell everyone to send a one- or two-second "dummy" upfront so the first song’s metadata gets shown.
Thanks for the pointer to #2138, maybe that can help.
@Moonbase59 Our users would roll back to LS 2.0.6 to avoid this issue, that might help you work around the problem for now.
@TheRealMattLear Thanks, that would be for production setups, yep. I’m currently evaluating AzuraCast and follow the latest Liquidsoap development with interest, so mine here is mainly a test & evaluation setup, of course pushing everything a little for testing and being sure about what I recommend to real station owners.
My testing streamers currently use a dummy 3s silence file when starting their transmissions, that helps avoid the problem and ensures the "real" show metadata is shown as soon as possible.
Hey @toots do you have any clue as to what the cause could be here? Sorry to pester, we have a few users asking us about a resolve
Thanks for the report. I'm gonna have a look very shortly.
And? Do you have any update?
Hey @toots sorry to ping again, just wondering if you have any updates on this issue and if there is anything we can do to help?
@toots If you’re working on this, please try with both Icecast "burst mode" enabled and disabled. In the past, we’ve sometimes seen problems with that feature (although it is a good feature and I usually enable it).
Hey @Moonbase59 would you be able to run a test and confirm if disabling Icecast burst mode helps the problem by any chance?
Hi all. I'm looking at this now. Do we have more steps to reproduce? A simple liquidsoap harbor with butt doesn't seem to reproduce it. Here's my script:
log.level := 4
s = playlist("/Users/toots/sources/test-stream/audio")
i = input.harbor("test")
s = fallback(track_sensitive=false, [i, s])
s = source.on_metadata(s, print)
output.ao(fallible=true, s)
When I connect with a BUTT client and the metadata set, I see the right sequence of actions: metadata being sent to the harbor buffer. Then wait for buffer to fill then metadata being printed.
Are y'all able to reproduce the issue with this script? If so, how and with which client?
As a first debug step, I would remove the output.dummy in this script. The operator is active so it will be drained of its content by the clock once it gets active.
I just pushed a commit that will start logging metadata being replayed when switching to the harbor. This should give us some indications. You will need to set the log level to 5
Thanks for looking into this @toots - it's been a tough one to replicate. We'll try removing output.dummy and seeing if the problem persists. Following that will install the custom commit.
There are definitely work around I could suggest but I would like to find a solution that works for most people.
I just pushed another small change to make the replay gain sources more robust.
Hi @toots thank you for getting back to this, I really appreciate it! I have tested removing output.dummy from the script but unfortunately that did not resolve the issue, I am still getting empty metadata. I turned my logging up to level 5 but it looks like those commits went to 2.2.X, I using 2.1.4, are you able to push those to the 2.1.X branch so that we can use them?