beets
beets copied to clipboard
beetbox
Spotify: Improve song search/matching
The Spotify plugin doesn't match/find most songs.
Problem
Running this command in verbose (-vv) mode:
$ beet -vv spotify
Led to this problem:
user configuration: /data/beets/config.yaml
data directory: /data/beets
plugin paths:
inline: adding item field first_artist
lastgenre: Loading canonicalization tree /etc/beets/genre-tree.yaml
lastgenre: Loading canonicalization tree /etc/beets/genre-tree.yaml
Sending event: pluginload
library database: /data/beets/library.db
library directory: /data/music
Sending event: library_opened
Parsed query: AndQuery([TrueQuery()])
Parsed sort: NullSort()
spotify: Processing 27 tracks...
spotify: Searching Spotify for 'Senegal Fast Food artist:Amadou & Mariam album:'
spotify: Found 0 result(s) from Spotify for 'Senegal Fast Food artist:Amadou & Mariam album:'
spotify: Searching Spotify for 'La Realite artist:Amadou & Mariam album:'
spotify: Found 0 result(s) from Spotify for 'La Realite artist:Amadou & Mariam album:'
spotify: Searching Spotify for 'Je pense a toi artist:Amadou & Mariam album:Sou Ni Tile'
spotify: Found 0 result(s) from Spotify for 'Je pense a toi artist:Amadou & Mariam album:Sou Ni Tile'
spotify: Searching Spotify for 'Deep Jungle Walk artist:Astrix album:'
spotify: Found 0 result(s) from Spotify for 'Deep Jungle Walk artist:Astrix album:'
spotify: Searching Spotify for 'Ephemeral artist:Bassnectar album:'
spotify: Found 0 result(s) from Spotify for 'Ephemeral artist:Bassnectar album:'
spotify: Searching Spotify for 'Street Blues artist:Bellaire album:'
spotify: Found 0 result(s) from Spotify for 'Street Blues artist:Bellaire album:'
spotify: Searching Spotify for 'Redemption artist:Boris Brejcha album:'
spotify: Found 0 result(s) from Spotify for 'Redemption artist:Boris Brejcha album:'
spotify: Searching Spotify for 'Vaunce of the Flowers (Waltz of the Flowers) artist:Tchaikovsky album:'
spotify: Found 0 result(s) from Spotify for 'Vaunce of the Flowers (Waltz of the Flowers) artist:Tchaikovsky album:'
spotify: Searching Spotify for 'Walkin' artist:Miles Davis album:Autumn Leaves'
spotify: Found 0 result(s) from Spotify for 'Walkin' artist:Miles Davis album:Autumn Leaves'
spotify: Searching Spotify for 'Autumn Leaves artist:Miles Davis album:Autumn Leaves'
spotify: Found 0 result(s) from Spotify for 'Autumn Leaves artist:Miles Davis album:Autumn Leaves'
spotify: Searching Spotify for 'So What artist:Miles Davis album:Autumn Leaves'
spotify: Found 0 result(s) from Spotify for 'So What artist:Miles Davis album:Autumn Leaves'
spotify: Searching Spotify for ''Round Midnight artist:Miles Davis album:Autumn Leaves'
spotify: Found 0 result(s) from Spotify for ''Round Midnight artist:Miles Davis album:Autumn Leaves'
spotify: Searching Spotify for 'All of You artist:Miles Davis album:Autumn Leaves'
spotify: Found 0 result(s) from Spotify for 'All of You artist:Miles Davis album:Autumn Leaves'
spotify: Searching Spotify for 'Ready or Not (Champion Bootleg) artist:Fugees album:'
spotify: Found 0 result(s) from Spotify for 'Ready or Not (Champion Bootleg) artist:Fugees album:'
spotify: Searching Spotify for 'Amanece artist:EttoreTechnoChannel album:'
spotify: Found 0 result(s) from Spotify for 'Amanece artist:EttoreTechnoChannel album:'
spotify: Searching Spotify for 'Free Tibet (Vini Vici remix) artist:Hilight Tribe album:'
spotify: Found 0 result(s) from Spotify for 'Free Tibet (Vini Vici remix) artist:Hilight Tribe album:'
spotify: Searching Spotify for 'Never Mind artist:Infected Mushroom album:Army of Mushrooms'
spotify: Found 2 result(s) from Spotify for 'Never Mind artist:Infected Mushroom album:Army of Mushrooms'
spotify: Most popular track chosen, count: 2
spotify: Searching Spotify for 'Franks artist:Infected Mushroom album:Legend of the Black Shawarma'
spotify: Found 1 result(s) from Spotify for 'Franks artist:Infected Mushroom album:Legend of the Black Shawarma'
spotify: Spotify track(s) found, count: 1
spotify: Searching Spotify for 'The Rope (Le Youth remix) feat. Polica artist:Lane 8 album:The Rope'
spotify: Found 1 result(s) from Spotify for 'The Rope (Le Youth remix) feat. Polica artist:Lane 8 album:The Rope'
spotify: Spotify track(s) found, count: 1
spotify: Searching Spotify for 'Mishaps Happening artist:Quantic album:'
spotify: Found 0 result(s) from Spotify for 'Mishaps Happening artist:Quantic album:'
spotify: Searching Spotify for 'The Ghost of Tom Joad artist:Rage Against The Machine album:'
spotify: Found 0 result(s) from Spotify for 'The Ghost of Tom Joad artist:Rage Against The Machine album:'
spotify: Searching Spotify for 'Wake Up (Rasticles drum n bass remix) artist:Rage Against the Machine album:'
spotify: Found 0 result(s) from Spotify for 'Wake Up (Rasticles drum n bass remix) artist:Rage Against the Machine album:'
spotify: Searching Spotify for 'The Alcoholic artist:Royksopp album:'
spotify: Found 0 result(s) from Spotify for 'The Alcoholic artist:Royksopp album:'
spotify: Searching Spotify for 'Back To Jazz artist:sometimes jah album:Just Feel'
spotify: Found 1 result(s) from Spotify for 'Back To Jazz artist:sometimes jah album:Just Feel'
spotify: Spotify track(s) found, count: 1
spotify: Searching Spotify for 'Blues artist:STAND HIGH PATROL album:'
spotify: Found 0 result(s) from Spotify for 'Blues artist:STAND HIGH PATROL album:'
spotify: Searching Spotify for 'STAND HIGH PATROL: Boat People artist:STAND HIGH PATROL album:'
spotify: Found 0 result(s) from Spotify for 'STAND HIGH PATROL: Boat People artist:STAND HIGH PATROL album:'
spotify: Searching Spotify for 'Lightweight artist:Total Science album:'
spotify: Found 0 result(s) from Spotify for 'Lightweight artist:Total Science album:'
spotify: 23 track(s) did not match a Spotify ID:
spotify: track: Senegal Fast Food artist:Amadou & Mariam album:
spotify: track: La Realite artist:Amadou & Mariam album:
spotify: track: Je pense a toi artist:Amadou & Mariam album:Sou Ni Tile
spotify: track: Deep Jungle Walk artist:Astrix album:
spotify: track: Ephemeral artist:Bassnectar album:
spotify: track: Street Blues artist:Bellaire album:
spotify: track: Redemption artist:Boris Brejcha album:
spotify: track: Vaunce of the Flowers (Waltz of the Flowers) artist:Tchaikovsky album:
spotify: track: Walkin' artist:Miles Davis album:Autumn Leaves
spotify: track: Autumn Leaves artist:Miles Davis album:Autumn Leaves
spotify: track: So What artist:Miles Davis album:Autumn Leaves
spotify: track: 'Round Midnight artist:Miles Davis album:Autumn Leaves
spotify: track: All of You artist:Miles Davis album:Autumn Leaves
spotify: track: Ready or Not (Champion Bootleg) artist:Fugees album:
spotify: track: Amanece artist:EttoreTechnoChannel album:
spotify: track: Free Tibet (Vini Vici remix) artist:Hilight Tribe album:
spotify: track: Mishaps Happening artist:Quantic album:
spotify: track: The Ghost of Tom Joad artist:Rage Against The Machine album:
spotify: track: Wake Up (Rasticles drum n bass remix) artist:Rage Against the Machine album:
spotify: track: The Alcoholic artist:Royksopp album:
spotify: track: Blues artist:STAND HIGH PATROL album:
spotify: track: STAND HIGH PATROL: Boat People artist:STAND HIGH PATROL album:
spotify: track: Lightweight artist:Total Science album:
spotify:
https://open.spotify.com/track/5qVNRfUrsJToWrIsdxytec
https://open.spotify.com/track/6zTbuAt90GFVTn3F651K5u
https://open.spotify.com/track/7toata3kAxgvAPer5XCwHE
https://open.spotify.com/track/0lYGEP22F5i29vvFZ8yuMW
Sending event: cli_exit
-> Only 4 of 27 songs matched (14%).
However, there should be a lot more matches, if not all. For instance Miles Davis' "Autumn Leaves" is definitely known by Spotify.
Something seems to be wrong with the way the Spotify search query is constructed. I briefly tried to run some of the generated queries against the Spotify API here and they returned proper results. Not sure why the Beets plugin isn't finding the songs. I suspect some difference in the way the Spotify query is constructed but I didn't spot it yet.
... On the other hand I did not configure Spotify authentication myself but when using the Spotify API on the documentation page, I was logged with my personal account. Is it possible that certain songs can be found only when logged in using a personal account?
Setup
- OS: Alpine 3.19 container within an Ubuntu 22.02 host.
- Python version: 3.12.2
- beets version: 1.6.1 + latest commits on main branch
- Turning off plugins made problem go away (yes/no): no
My configuration (output of beet config) is:
beet config
beet config
importfeeds:
relative_to: !!binary |
L2RhdGEvaW1wb3J0ZmVlZA==
formats: m3u
dir: /data/importfeed
m3u_name: imported.m3u
absolute_path: no
directory: /data/music
# --------------- Main ---------------
library: /data/beets/library.db
paths:
default: Albums/%title{$albumartist}/$album%aunique{}/$track $title
singleton: Singles/%title{$first_artist}/$title
comp: Compilations/$album%aunique{}/$track $title
# --------------- Plugins ---------------
plugins:
- describe
- web
- webm3u
- webrouter
- beetstream
- types
- smartplaylist
- importfeeds
- edit
- play
- mbsync
- spotify
- fromfilename
- lastgenre
- autogenre
- ftintitle
- inline
- rewrite
- ihate
- chroma
- xtractor
- mpdstats
- convert
- bpm
- duplicates
- bareasc
- fuzzy
- info
- ytimport
- badfiles
- unimported
- random
webrouter:
host: 0.0.0.0
port: 8337
routes:
/:
plugin: web
/favicon.ico:
plugin: webrouter.favicon
/subsonic:
plugin: beetstream
config:
never_transcode: yes
playlist_dir: /data/playlists
/m3u:
plugin: webm3u
cors: ''
cors_supports_credentials: no
reverse_proxy: no
include_paths: no
readonly: yes
beetstream:
host: 0.0.0.0
port: 8080
never_transcode: yes
cors: '*'
cors_supports_credentials: yes
reverse_proxy: no
include_paths: no
playlist_dir: ''
web:
host: 0.0.0.0
port: 8337
reverse_proxy: yes
readonly: yes
cors: ''
cors_supports_credentials: no
include_paths: no
webm3u:
host: 0.0.0.0
port: 8339
cors: ''
cors_supports_credentials: no
reverse_proxy: no
include_paths: no
playlist_dir:
uri_format:
import:
write: yes
copy: yes
move: no
link: no
hardlink: no
incremental: yes
quiet: no
quiet_fallback: skip
log: /data/beets/import.log
ytimport:
directory: /data/ytimport
import: yes
reimport: no
format: bestaudio/best
url_file: ''
likes: no
max_likes: 15
auth_headers:
set: {}
min_length: 60
max_length: 7200
max_length_nochapter: 900
split_tracks: yes
group_albums: yes
spotify:
source_weight: 0.7
mode: list
tiebreak: popularity
show_failures: yes
artist_field: albumartist
album_field: album
track_field: title
region_filter:
regex: []
client_id: 4e414367a1d14c75a5c5129a627fcab8
client_secret: REDACTED
tokenfile: spotify_token.json
xtractor:
auto: no
dry-run: no
write: yes
threads: 4
force: no
quiet: no
keep_output: no
keep_profiles: no
output_path: /tmp/xtractor
essentia_extractor: /usr/local/bin/essentia_streaming_extractor_music
high_level_targets:
genre_rosamerica_probability:
path: highlevel.genre_rosamerica.probability
type: float
genre_electronic:
path: highlevel.genre_electronic.value
type: string
genre_electronic_probability:
path: highlevel.genre_electronic.probability
type: float
timbre:
path: highlevel.timbre.value
type: string
tonal_atonal:
path: highlevel.tonal_atonal.value
type: string
key_edma:
path: tonal.key_edma.key
type: string
key_edma_scale:
path: tonal.key_edma.scale
type: string
danceable:
path: highlevel.danceability.all.danceable
type: float
required: yes
gender:
path: highlevel.gender.value
type: string
required: yes
is_male:
path: highlevel.gender.all.male
type: float
is_female:
path: highlevel.gender.all.female
type: float
genre_rosamerica:
path: highlevel.genre_rosamerica.value
type: string
required: yes
voice_instrumental:
path: highlevel.voice_instrumental.value
type: string
required: yes
is_voice:
path: highlevel.voice_instrumental.all.voice
type: float
is_instrumental:
path: highlevel.voice_instrumental.all.instrumental
type: float
mood_acoustic:
path: highlevel.mood_acoustic.all.acoustic
type: float
required: yes
mood_aggressive:
path: highlevel.mood_aggressive.all.aggressive
type: float
required: yes
mood_electronic:
path: highlevel.mood_electronic.all.electronic
type: float
required: yes
mood_happy:
path: highlevel.mood_happy.all.happy
type: float
required: yes
mood_sad:
path: highlevel.mood_sad.all.sad
type: float
required: yes
mood_party:
path: highlevel.mood_party.all.party
type: float
required: yes
mood_relaxed:
path: highlevel.mood_relaxed.all.relaxed
type: float
required: yes
mood_mirex:
path: highlevel.moods_mirex.value
type: string
required: yes
mood_mirex_cluster_1:
path: highlevel.moods_mirex.all.Cluster1
type: float
mood_mirex_cluster_2:
path: highlevel.moods_mirex.all.Cluster2
type: float
mood_mirex_cluster_3:
path: highlevel.moods_mirex.all.Cluster3
type: float
mood_mirex_cluster_4:
path: highlevel.moods_mirex.all.Cluster4
type: float
mood_mirex_cluster_5:
path: highlevel.moods_mirex.all.Cluster5
type: float
extractor_profile:
highlevel:
svm_models:
- /var/lib/essentia/svm-models/beta5/danceability.history
- /var/lib/essentia/svm-models/beta5/gender.history
- /var/lib/essentia/svm-models/beta5/genre_rosamerica.history
- /var/lib/essentia/svm-models/beta5/genre_electronic.history
- /var/lib/essentia/svm-models/beta5/mood_acoustic.history
- /var/lib/essentia/svm-models/beta5/mood_aggressive.history
- /var/lib/essentia/svm-models/beta5/mood_electronic.history
- /var/lib/essentia/svm-models/beta5/mood_happy.history
- /var/lib/essentia/svm-models/beta5/mood_sad.history
- /var/lib/essentia/svm-models/beta5/mood_party.history
- /var/lib/essentia/svm-models/beta5/mood_relaxed.history
- /var/lib/essentia/svm-models/beta5/moods_mirex.history
- /var/lib/essentia/svm-models/beta5/voice_instrumental.history
- /var/lib/essentia/svm-models/beta5/tonal_atonal.history
- /var/lib/essentia/svm-models/beta5/timbre.history
compute: 1
outputFormat: json
outputFrames: 0
lowlevel:
frameSize: 2048
hopSize: 1024
chromaprint:
compute: 0
low_level_targets:
bpm:
path: rhythm.bpm
type: integer
required: yes
danceability:
path: rhythm.danceability
type: float
beats_count:
path: rhythm.beats_count
type: integer
average_loudness:
path: lowlevel.average_loudness
type: float
required: yes
ftintitle:
auto: yes
drop: no
format: feat. {0}
item_fields:
first_artist: "import re\nm = re.match('^(.+?)\\\\s+(x\\\\s|\\\\(?(feat(\\\\.?|uring)|(ft|vs)\\\\.))', artist, flags=re.IGNORECASE)\nif m:\n return m[1]\nreturn artist\n"
bareasc:
prefix: '#'
fuzzy:
threshold: 0.7
prefix: '~'
chroma:
auto: yes
autogenre:
pretend: no
all: no
force: no
lastgenre: yes
xtractor: yes
remix_title: yes
parent_genres: yes
from_title: yes
genre_rosamerica_strong: 0.8
genre_electronic_strong: 0.8
genre_electronic_prepend: 0.5
genre_electronic_append: 0.45
lastgenre:
auto: no
prefer_specific: yes
count: 4
source: album
min_weight: 15
canonical: /etc/beets/genre-tree.yaml
whitelist: /data/beets/genres.txt
fallback:
force: yes
separator: ', '
title_case: yes
ihate:
warn: ['artist:rnb']
skip:
- genre:Comedy
- artist::^Adele$
- artist::^AYLIVA$
- artist::^Sia$
convert:
never_convert_lossy_files: yes
embed: no
hardlink: no
dest:
pretend: no
link: no
threads: 16
format: mp3
id3v23: inherit
formats:
aac:
command: ffmpeg -i $source -y -vn -acodec aac -aq 1 $dest
extension: m4a
alac:
command: ffmpeg -i $source -y -vn -acodec alac $dest
extension: m4a
flac: ffmpeg -i $source -y -vn -acodec flac $dest
mp3: ffmpeg -i $source -y -vn -aq 2 $dest
opus: ffmpeg -i $source -y -vn -acodec libopus -ab 96k $dest
ogg: ffmpeg -i $source -y -vn -acodec libvorbis -aq 3 $dest
wma: ffmpeg -i $source -y -vn -acodec wmav2 -vn $dest
max_bitrate:
auto: no
auto_keep: no
tmpdir:
quiet: no
paths: {}
no_convert: ''
copy_album_art: no
album_art_maxwidth: 0
delete_originals: no
playlist:
duplicates:
delete: no
album: no
checksum: ''
copy: ''
count: no
format: ''
full: no
keys: []
merge: no
move: ''
path: no
tiebreak: {}
strict: no
tag: ''
bpm:
max_strokes: 3
overwrite: yes
play:
command: play-playlist
use_folders: no
relative_to:
raw: no
warning_threshold: 100
bom: no
smartplaylist:
auto: no
output: extm3u
fields: [id]
playlist_dir: /data/playlists
relative_to: /data/playlists
playlists:
- name: all.m3u
query: ''
- name: all-by-album-genre.m3u
query: albumartist+ year+ album+ disc+ track+ genre+ artist+ title+
- name: jazz-blues.m3u
query:
- genre::^Jazz
- genre::^Blues
- genre:'Jazz Fusion'
- genre:'Swing Jazz'
- name: reggae.m3u
query: ['genres:Reggae', 'genres::''^Dub$''', 'genres:Ska']
- name: dub.m3u
query: ['genre::''^Dub$''']
- name: ambient.m3u
query: ['genre:Ambient']
- name: chillout.m3u
query:
- genre:Ambient
- genre:Downtempo
- genre:'Nu Jazz'
- genre:'Trip Hop'
- genre:Electronica
- genre::'^Electro$'
- genre:'Easy Listening'
- name: idm.m3u
query: ['genre:Idm', 'genre:Breakcore']
- name: downtempo.m3u
query: ['genre:Downtempo', 'genre:''Nu Jazz''']
- name: nu-jazz.m3u
query: ['genre:''Nu Jazz''']
- name: world-music.m3u
query:
- genre:Blues
- genres:African
- genres:Afrobeat
- genres:'Rhythm And Blues'
- genres::'(^|, )Funk(,|$)'
- genres::'(^|, )Soul(,|$)'
- genres:'Caribbean And Latin American'
- genre:Brazilian
- genre:Balkan
- genre:Klezmer
- genres:'East European'
- name: african.m3u
query: ['genres:African', 'genres:Afrobeat']
- name: afrobeat.m3u
query: ['genres:Afrobeat']
- name: balkan-klezmer.m3u
query: ['genre:''East European''', 'genre:Balkan', 'genre:Klezmer']
- name: classical.m3u
query: ['genre:Classical']
- name: dnb.m3u
query:
- genres:'Drum And Bass'
- genre:Breakbeat
- genre:Jungle
- genre:Neurofunk
- genre:'Liquid Drum And Bass'
- genre:'Ragga Drum And Bass'
- name: neurofunk.m3u
query: ['genre:Neurofunk']
- name: jungle.m3u
query: ['genre:Jungle']
- name: uk-garage.m3u
query: ['genres:''Uk Garage''', 'genres:Dubstep', 'genre:Grime']
- name: dubstep-grime.m3u
query: ['genre:''Dark Dubstep''', 'genre:Dubstep bpm:0..90', 'genre:Grime']
- name: electronic.m3u
query: ['genres:Electronic']
- name: house-techno.m3u
query:
- genre:House
- genre:Techno
- genre:Minimal
- genre:Progressive
- name: deep-house.m3u
query: ['genre:''Deep House''', 'genre:''Nu Jazz'' genres:House']
- name: house.m3u
query: ['genre:House', 'genre:''Nu Jazz'' genres:House']
- name: tech-house-minimal.m3u
query: ['genre:''Tech House''', 'genre:Minimal']
- name: techno.m3u
query: ['genre:Techno', 'genre:Minimal']
- name: minimal.m3u
query: ['genre:Minimal']
- name: trance.m3u
query: ['genre:Trance']
- name: goa.m3u
query: ['genre:''Goa Trance''']
- name: ska.m3u
query: ['genre:Ska']
- name: rock.m3u
query:
- genre::'^Rock'
- genre::'^New Wave'
- genre::'^Folk'
- genre:'Fun Metal'
- name: punk-rock.m3u
query: ['genre:Punk', 'genre:''Fun Metal''']
- name: pop.m3u
query:
- genre:Pop
- genre:'New Wave'
- genre:'Balkan Beats'
- genre::'^Electro$'
- genres::'^Electronic Rock, (?!Drum And Bass)'
- name: hiphop.m3u
query: ['genre:''Hip Hop''']
- name: liked/1 or more.m3u
query: play_count:1..
- name: liked/5 or more.m3u
query: play_count:5..
- name: liked/loved.m3u
query: loved:1
- name: latest/last day.m3u
query: added- added:-1d..
- name: latest/last 3 days.m3u
query: added- added:-3d..
- name: latest/last 7 days.m3u
query: added- added:-7d..
- name: latest/last 14 days.m3u
query: added- added:-14d..
- name: latest/last 30 days.m3u
query: added- added:-30d..
- name: latest/last 60 days.m3u
query: added- added:-60d..
uri_format:
forward_slash: no
prefix: REDACTED
urlencode: no
pretend_paths: no
unimported:
ignore_extensions: []
ignore_subdirectories: []
rewrite: {}
pathfields: {}
album_fields: {}
mpd:
music_directory: /data/music
strip_path: ''
rating: yes
rating_mix: 0.75
host: localhost
port: 6600
password: REDACTED
edit:
albumfields: album albumartist
itemfields: track title artist album
ignore_fields: id path
describe:
auto: no
I have had similar experiences. As discussed in #4997, we should move to the Spotipy library, which does a great job of interfacing with Spotify.
I believe there's a bug in spotify.py. The query needs to be URL-encoded twice, which is a temporary fix rather than a proper solution. I don't intend to submit a PR for this, as I think using the Spotify library would be a better approach.
