liquidsoap
liquidsoap copied to clipboard
savonet
Playlist "randomize" mode doesn't behave very random in the first few hours after a liquidsoap restart or playlist reload.
Describe the bug
I've repeatedly noticed lately (probably since version 2.0.0, although i'm not entirely sure at what exact version the problem started - it was certainly an issue with 2.0.2 and has stayed the same after updating to 2.0.6) that playlist(mode="randomize") starts out not very random after (re-)loading the playlist. Instead there are clear patterns and unusually frequent artist repetitions going on. So far, whenever i've checked again the next day, things seemed to have calmed down by then and the song picks felt reasonably random again. To give you an impression of the problem, here are some log excerpts.
The "rotation" playlist is the wildest example here. You'll sure notice that each track played here has a filename starting with a letter in the M-Z range. Weird stuff! Plenty of files starting with m, n, o, p, - then there are few-to-none with r, s, t... then we're going wild in the u-z range again. Also, there are way more artist repetitions than you'd expect from a directory containing roughly 17500 tracks. Since this is the largest of my three playlists, it sure looks like this problem gets triggered more heavily when loading large playlists/directories.
2022/07/09 16:13:57 [rotation:3] Reloading playlist with URI /home/pi/music/rotation.
2022/07/09 16:15:44 [rotation:3] Prepared "/home/pi/music/rotation/Print Head - I'mABug(Urinals).flac" (RID 5).
2022/07/09 16:24:44 [rotation:3] Prepared "/home/pi/music/rotation/USA Nails - Temporary Home.flac" (RID 11).
2022/07/09 16:27:39 [rotation:3] Prepared "/home/pi/music/rotation/Ukryte Zalety Systemu - Tv Trwaj.flac" (RID 13).
2022/07/09 16:32:31 [rotation:3] Prepared "/home/pi/music/rotation/Protomartyr - Ellen.flac" (RID 15).
2022/07/09 16:39:32 [rotation:3] Prepared "/home/pi/music/rotation/Sauna Youth - Unreal City.flac" (RID 17).
2022/07/09 16:42:45 [rotation:3] Prepared "/home/pi/music/rotation/Nutrition - Medicated Sleep.flac" (RID 20).
2022/07/09 16:46:21 [rotation:3] Prepared "/home/pi/music/rotation/Warm Bodies - I Smashed It.flac" (RID 21).
2022/07/09 16:49:32 [rotation:3] Prepared "/home/pi/music/rotation/Ursula - Relief.flac" (RID 26).
2022/07/09 16:52:13 [rotation:3] Prepared "/home/pi/music/rotation/Oily Boys - Headstone.flac" (RID 28).
2022/07/09 16:56:53 [rotation:3] Prepared "/home/pi/music/rotation/Nuts - Art-School-Yuppies Die!.flac" (RID 32).
2022/07/09 17:03:26 [rotation:3] Prepared "/home/pi/music/rotation/Weenog - Curse.flac" (RID 34).
2022/07/09 17:05:44 [rotation:3] Prepared "/home/pi/music/rotation/Molar - Auslännerwahlrecht.flac" (RID 35).
2022/07/09 17:09:26 [rotation:3] Prepared "/home/pi/music/rotation/Modern Needs - Upward Mobility.flac" (RID 36).
2022/07/09 17:15:41 [rotation:3] Prepared "/home/pi/music/rotation/We the Pigs - It Didn´t Hurt.flac" (RID 40).
2022/07/09 17:18:12 [rotation:3] Prepared "/home/pi/music/rotation/Vaguess - Ran to You.flac" (RID 43).
2022/07/09 17:21:49 [rotation:3] Prepared "/home/pi/music/rotation/Nuts - Children Of The Devolution.flac" (RID 45).
2022/07/09 17:23:50 [rotation:3] Prepared "/home/pi/music/rotation/Video Duct - President Nawaz.flac" (RID 47).
2022/07/09 17:35:03 [rotation:3] Prepared "/home/pi/music/rotation/Vidro - Hans Tid är Slut.flac" (RID 52).
2022/07/09 17:36:49 [rotation:3] Prepared "/home/pi/music/rotation/Neo Neos - Submarine Races v1.flac" (RID 53).
2022/07/09 17:41:03 [rotation:3] Prepared "/home/pi/music/rotation/Vision Demon - But you.flac" (RID 55).
2022/07/09 17:48:54 [rotation:3] Prepared "/home/pi/music/rotation/Vanity - Pounding Boogie.flac" (RID 56).
2022/07/09 17:55:36 [rotation:3] Prepared "/home/pi/music/rotation/Uffizi - Sunflower.flac" (RID 61).
2022/07/09 17:57:44 [rotation:3] Prepared "/home/pi/music/rotation/Mini Skirt - Brigantine St.flac" (RID 62).
2022/07/09 18:01:17 [rotation:3] Prepared "/home/pi/music/rotation/Modern Needs - Ineptitude.flac" (RID 64).
2022/07/09 18:06:21 [rotation:3] Prepared "/home/pi/music/rotation/Monoburro - Ignorante.flac" (RID 69).
2022/07/09 18:08:03 [rotation:3] Prepared "/home/pi/music/rotation/UV-TV - Glass.flac" (RID 72).
2022/07/09 18:11:50 [rotation:3] Prepared "/home/pi/music/rotation/Omi Palone - Follow Your Visions.flac" (RID 75).
2022/07/09 18:15:03 [rotation:3] Prepared "/home/pi/music/rotation/Neo Neos - Stupid Symbol.flac" (RID 76).
2022/07/09 18:16:07 [rotation:3] Prepared "/home/pi/music/rotation/Mini Skirt - Farkurnell.flac" (RID 79).
2022/07/09 18:18:06 [rotation:3] Prepared "/home/pi/music/rotation/Omi Palone - Shallow Divide.flac" (RID 80).
2022/07/09 18:21:33 [rotation:3] Prepared "/home/pi/music/rotation/VHS - Public Act.flac" (RID 81).
2022/07/09 18:25:18 [rotation:3] Prepared "/home/pi/music/rotation/Pallet - Consumption.flac" (RID 84).
2022/07/09 18:27:04 [rotation:3] Prepared "/home/pi/music/rotation/Negative Scanner - The Only One.flac" (RID 85).
2022/07/09 18:28:50 [rotation:3] Prepared "/home/pi/music/rotation/Nutrition - Out Of Time.flac" (RID 86).
2022/07/09 18:32:50 [rotation:3] Prepared "/home/pi/music/rotation/Ubik - New Disease.flac" (RID 87).
2022/07/09 18:34:40 [rotation:3] Prepared "/home/pi/music/rotation/Monoburro - Cyborg.flac" (RID 89).
2022/07/09 18:39:43 [rotation:3] Prepared "/home/pi/music/rotation/Nuts - Lobster Roll.flac" (RID 92).
2022/07/09 18:41:57 [rotation:3] Prepared "/home/pi/music/rotation/Nuclear Moms - Tornado Flower Garden.flac" (RID 93).
2022/07/09 18:45:10 [rotation:3] Prepared "/home/pi/music/rotation/Vexx - Roky, Take Me Home.flac" (RID 95).
2022/07/09 18:46:43 [rotation:3] Prepared "/home/pi/music/rotation/Wash - Goofy's Concern.flac" (RID 96).
2022/07/09 18:48:57 [rotation:3] Prepared "/home/pi/music/rotation/Videodrome - Metro Station.flac" (RID 98).
The "classics" playlist (containing ~8000 tracks) ain't quite as bad but you can notice obvious patterns and artist repetitions here too... in addition to an unlikely abundance of tracks whose filenames start with "De..." or "Di...".
2022/07/09 16:13:37 [classics:3] Reloading playlist with URI /home/pi/music/classics.
2022/07/09 16:14:18 [classics:3] Prepared "/home/pi/music/classics/The Sound - Unwritten Law.flac" (RID 4).
2022/07/09 16:22:36 [classics:3] Prepared "/home/pi/music/classics/Poison Idea - New Right.flac" (RID 6).
2022/07/09 16:40:19 [classics:3] Prepared "/home/pi/music/classics/Savage Beliefs - Double Standard.flac" (RID 7).
2022/07/09 16:45:01 [classics:3] Prepared "/home/pi/music/classics/Radio Birdman - Time to Fall.flac" (RID 8).
2022/07/09 16:55:21 [classics:3] Prepared "/home/pi/music/classics/Wreck - Raised By Whites.flac" (RID 9).
2022/07/09 16:58:56 [classics:3] Prepared "/home/pi/music/classics/Mars - Helen Fordsdale.flac" (RID 18).
2022/07/09 17:01:23 [classics:3] Prepared "/home/pi/music/classics/Minutemen - Games.flac" (RID 29).
2022/07/09 17:04:25 [classics:3] Prepared "/home/pi/music/classics/Didjits - Fix Some Food Bitch.flac" (RID 33).
2022/07/09 17:07:58 [classics:3] Prepared "/home/pi/music/classics/Earth Mother Fucker - Teenage Pricks.flac" (RID 37).
2022/07/09 17:28:37 [classics:3] Prepared "/home/pi/music/classics/Die Kreuzen - Don't Say Please.flac" (RID 41).
2022/07/09 17:29:50 [classics:3] Prepared "/home/pi/music/classics/Didjits - Hafta Be Cool To Rule-Wingtips.flac" (RID 42).
2022/07/09 17:39:13 [classics:3] Prepared "/home/pi/music/classics/Laughing Hyenas - Candy.flac" (RID 44).
2022/07/09 17:46:57 [classics:3] Prepared "/home/pi/music/classics/Didjits - Lou Reed.flac" (RID 46).
2022/07/09 18:05:41 [classics:3] Prepared "/home/pi/music/classics/MIA - Small Man In A Big World.flac" (RID 57).
2022/07/09 18:22:31 [classics:3] Prepared "/home/pi/music/classics/Dead Moon - Castaways.flac" (RID 60).
2022/07/09 18:31:51 [classics:3] Prepared "/home/pi/music/classics/Live Skull - Sparky.flac" (RID 63).
2022/07/09 18:38:29 [classics:3] Prepared "/home/pi/music/classics/Mule - X & 29.flac" (RID 68).
2022/07/09 18:54:03 [classics:3] Prepared "/home/pi/music/classics/Dead Kennedys - I Kill Children.flac" (RID 78).
The "new" playlist is the shortest here (~1600 tracks) but shows the same familiar weirdness:
2022/07/09 16:13:47 [new:3] Reloading playlist with URI /home/pi/music/new.
2022/07/09 16:18:58 [new:3] Prepared "/home/pi/music/new/Real People - I Wanna go Home.flac" (RID 3).
2022/07/09 16:23:42 [new:3] Prepared "/home/pi/music/new/Print Head - Daily Living.flac" (RID 10).
2022/07/09 16:33:10 [new:3] Prepared "/home/pi/music/new/Ädwud - The Thom Yorke Remembrance Parade.flac" (RID 12).
2022/07/09 17:10:36 [new:3] Prepared "/home/pi/music/new/Vidro - Dö För Ingenting.flac" (RID 14).
2022/07/09 17:13:55 [new:3] Prepared "/home/pi/music/new/Skid City - Twister.flac" (RID 16).
2022/07/09 17:20:03 [new:3] Prepared "/home/pi/music/new/Split System - Hoping.flac" (RID 19).
2022/07/09 17:44:20 [new:3] Prepared "/home/pi/music/new/Red Gaze - Narcissistic Injury.flac" (RID 27).
2022/07/09 17:50:44 [new:3] Prepared "/home/pi/music/new/Tetanus - Nahschiss.flac" (RID 48).
2022/07/09 17:51:46 [new:3] Prepared "/home/pi/music/new/Split System - Climbing.flac" (RID 51).
2022/07/09 18:03:51 [new:3] Prepared "/home/pi/music/new/Tetanus - Borderline.flac" (RID 54).
2022/07/09 19:00:13 [new:3] Prepared "/home/pi/music/new/Spodee Boy - Barn Burner.flac" (RID 67).
To Reproduce I can't really tell what's actually triggering the problem. This is the script i'm running:
settings.log.file.path.set("/home/pi/music/log/12xu.log")
settings.log.file.set(true)
settings.log.file.append.set(true)
settings.encoder.metadata.export.set(["artist","title","album","date","year"])
settings.scheduler.non_blocking_queues.set(6)
settings.scheduler.generic_queues.set(6)
settings.scheduler.fast_queues.set(6)
settings.scheduler.log.set(false)
settings.server.log.level.set(4)
settings.server.telnet.set(true)
settings.server.telnet.port.set(1234)
settings.server.telnet.bind_addr.set("127.0.0.1")
settings.decoder.decoders.set(["FFMPEG","FLAC"])
settings.server.timeout.set(300.)
settings.init.daemon.set(true)
settings.init.daemon.pidfile.set(true)
settings.init.daemon.pidfile.path.set("/home/pi/12xu.pid")
audio.samplerate.set(44100)
audio.channels.set(2)
classics = playlist(id="classics",mode="randomize",reload=1,reload_mode="rounds",prefetch=5,"/home/pi/music/classics")
rotation = playlist(id="rotation",mode="randomize",reload=1,reload_mode="rounds",prefetch=5,"/home/pi/music/rotation")
new_stuff = playlist(id="new",mode="randomize",reload=1,reload_mode="rounds",prefetch=5,"/home/pi/music/new")
music = random(id="music",weights=[1,1,2],[classics,new_stuff,rotation])
music.on_metadata(lastfm.submit(
user="*********",
password="********")
)
def librefm.submit(~user,~password,~source="broadcast",~length=false,m) =
audioscrobbler.submit(user=user,password=password,
source=source,length=length,
host="turtle.libre.fm",port=80,
m)
end
def listenbrainz(~user,~password,~source="broadcast",~length=false,m) =
audioscrobbler.submit(user=user,password=password,
source=source,length=length,
host="proxy.listenbrainz.org",port=80,
m)
end
music.on_metadata(librefm.submit(
source="broadcast",
user="********",
password="********"),
)
music.on_metadata(listenbrainz(
source="broadcast",
user="********",
password="********")
)
music2 = music
# A function applied to each metadata chunk
def append_title(m) =
# Grab the current title
title = m["title"]
date = m["date"]
# Return a new title metadata
[("title","#{title} (#{date})")]
end
# Apply map_metadata to s using append_title
music2 = map_metadata(append_title, music2)
ads1 = request.queue(id="ads1")
ads_music = fallback([ads1,music2])
def default_fade_values(m) =
# Set default fade_in if not present
m =
if m["liq_cross_duration"] == "" then
list.add(("liq_cross_duration","3.5"),m)
else
m
end
m
end
ads_music = map_metadata(default_fade_values,ads_music)
crf = crossfade(conservative=true,ads_music)
oops = blank(duration=-1.)
full=fallback([crf,oops],track_sensitive=false)
output.icecast(
%mp3.vbr(
quality=2,
min_bitrate=128,
max_bitrate=320),
icy_metadata="true",
fallible=true,
on_error=fun(_)->60.,
host="**.**.laut.fm",
port=8080,
password="********",
mount="/********",
full)
Note: I set prefetch=5 purely for testing purposes. The behaviour was the same when i left out the prefetch option.
The hardware is a 8 GB raspberry pi 4. Debian/RaspberryOS is booted from an external SSD connected via USB 3. The audio files are stored on an external hard drive, also connected via USB 3.
This is all i can give you to work with right now. Please let me know what other kind of more verbose or specific log data would be helpful to narrow things down.
As far as i understand, mode=randomize generates a full shuffled sequence of tracks when the playlist is loaded, right? Is there a way to make liquidsoap output the full track order?
Expected behavior
You'd expect the playback order to look and feel reasonably random.
Version details
- OS: Debian 11 (Bullseye) aka Raspbian / Raspberry Pi OS
Install method opam
Maybe related to #2057?
Sounds likely, although i haven't noticed any actual duplicates being played. Might be sheer luck though - i'm gonna have to comb through last week's log files a bit to see if that's the case.
edit I went through the logs and couldn't find any smoking gun duplicates between re-loads or liquidsoap restarts. Guess i'm just gonna wait for a 2.1.x release to see if the problem still persists then. Thanks for the superfast, hopefully successful fix. Fingers crossed. :smile:
After I had been observing that strange "randomize" behavior for a long time too, I've been testing the relevant patch for 2.1.0 with the modified list.liq (commit ac4233e) in 2.0.6. It looks quite good so far.
Since 2.0.7 should be released soon, I wonder why this patch isn't planned there so far...
This is my fault, I forgot to add it to the realease... I just did it in f2a87e7a906464a89bd1bf7917d7b5269d6039ce. As you can see the diff is real small and easy to implement on your local version in the meantime.