jellyseerr icon indicating copy to clipboard operation
jellyseerr copied to clipboard

feat: Added music support through Lidarr

Open 0-Pierre opened this issue 1 year ago • 54 comments

Description

Jellyseerr fully support music trough Lidarr 🚀 🎵

Metadata We’re using the Lidarr API (api.lidarr.audio) to improve speed, as MusicBrainz (MB) often suffers from high latency and server overload issues. Trade-off: The results are slightly less precise compared to the MB Search API.

Overview The overview section pulls data from Wikipedia, ensuring multilingual support for more flexible and accurate descriptions.

Image Handling Images are initially loaded from the provided links and cached locally. The Lidarr API offers additional artist images that aren’t available in the Cover Art Archive (CAA). If an image is missing, the system automatically falls back to CAA.

Caching By default, images from CAA and Fanart are cached locally, as these sources tend to load slowly. Cached images are stored in the /cache folder and can be cleared via the settings page using a dedicated job. Other images are only cached if specific options are enabled.

Similar Artists / Trending We use the ListenBrainz API to provide recommendations for similar artists and trending music. Missing images are completed through the Lidarr API, with a fallback to CAA.

Artist Groups The /group endpoint is dedicated to artist groups (bands). For individual artists (persons), we merge data with the /person endpoint from TMDb to avoid duplicate entries.

Informations Points There are cases where images aren’t available on either the CAA or the Lidarr API. In such instances, we need to upload them to the MusicBrainz database to ensure they are displayed. This aligns with the core purpose of MusicBrainz, a community-driven platform for sharing metadata. For this reason, I’ve removed Deezer as a fallback. Encouraging community contributions is more in line with MusicBrainz’s philosophy. Additionally, using Deezer’s fallback wasn’t always reliable, as the matching wasn’t based on precise IDs, though it worked correctly about 95% of the time.

Screenshot (if UI-related)

localhost_5055_ (1) localhost_5055_ (2) localhost_5055_ (4) localhost_5055_ (3)

To-Dos

  • [ ] Successful build pnpm build
  • [ ] Translation keys pnpm i18n:extract
  • [ ] Database migration (if required)

Issues Fixed or Closed

  • Closed:#96

0-Pierre avatar Jan 08 '25 20:01 0-Pierre

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

github-actions[bot] avatar Jan 09 '25 23:01 github-actions[bot]

@0-Pierre do not merge develop into this PR. Rebase on develop instead. Merging develop is a very bad practice, it makes things harder for us to review and later to track changes. It is written as well in our contribution guide.

Could you please rebase and remove all theses merge commits?

gauthier-th avatar Jan 10 '25 20:01 gauthier-th

@0-Pierre do not merge develop into this PR. Rebase on develop instead. Merging develop is a very bad practice, it makes things harder for us to review and later to track changes. It is written as well in our contribution guide.

Could you please rebase and remove all theses merge commits?

Apologies, I'm not very familiar with the GitHub processes and features. I'll do my best, but the first commit was automatically created during a rebase when I used the "Rebase current branch..." option.

0-Pierre avatar Jan 10 '25 21:01 0-Pierre

A preview for this feature is available as preview-music-support tag.

(In about 15-20 mins)

fallenbagel avatar Jan 14 '25 06:01 fallenbagel

Hi! I briefly tested the PR. Great work! Some feedbacks (Warning! I'm quite a noob, so they might be wrong or due to inexperience):

  1. If Jellyseerr is already configured and you try to request any kind of music related content, you get a generic error. The fact that Lidarr was not configured should have been highlighted in the popup error, in my opinion.
  2. If an album doesn't have a cover, it is shown as it is hovered with the mouse, I think it's a bit odd.
  3. When an album is displayed, it contains a certain number of tracks (I suppose disk 1). When it is then selected in Lidarr, it might contain a whole lot of other tracks. You can test with Green Day - American Idiot: in Jellyseerr 13 tracks are listed, in Lidarr they are 57.
  4. There is no filter in the music panel. Is this just because it wasn't implemented yet or are there problems?
  5. If an artist is not monitored in Lidarr and I request an album from said artist, the artist gets added, and it takes a lot (>1 min, nothing unbearable, but I noticed it) before the album is monitored and then searched. Is this wanted for any reason?

Thank you again for the work!

AlessandroTischer avatar Jan 14 '25 09:01 AlessandroTischer

Hi! I briefly tested the PR. Great work! Some feedbacks (Warning! I'm quite a noob, so they might be wrong or due to inexperience):

  1. If Jellyseerr is already configured and you try to request any kind of music related content, you get a generic error. The fact that Lidarr was not configured should have been highlighted in the popup error, in my opinion.
  2. If an album doesn't have a cover, it is shown as it is hovered with the mouse, I think it's a bit odd.
  3. When an album is displayed, it contains a certain number of tracks (I suppose disk 1). When it is then selected in Lidarr, it might contain a whole lot of other tracks. You can test with Green Day - American Idiot: in Jellyseerr 13 tracks are listed, in Lidarr they are 57.
  4. There is no filter in the music panel. Is this just because it wasn't implemented yet or are there problems?
  5. If an artist is not monitored in Lidarr and I request an album from said artist, the artist gets added, and it takes a lot (>1 min, nothing unbearable, but I noticed it) before the album is monitored and then searched. Is this wanted for any reason?

Thank you again for the work!

@AlessandroTischer Hey! Thanks for your notes, here's some points : 1- I'm fairly certain that the exact same behavior applies to Movies and Shows with Sonarr and Radarr 2 - The behavior is consistent across Movies and Show cards without covers, but missing covers are more frequent for Music, making it more noticeable. I can implement a fix to prevent the 'hovered' style from being applied to any card (Movies, TV, or Music) when a cover is missing. 3 - This occurs because I haven't configured the same release selection method used by Lidarr, such as choosing the best metadata or the most recent release. The current implementation is still very basic. 4 - Exactly, I’ve only configured the sorting options and nothing beyond that. 5 - Here's how the system currently works: if the artist doesn't already exist, it's first created in Lidarr without monitoring any of their albums. We then give Lidarr time to fetch all the metadata for the artist's discography. After about 40 seconds, the requested album is set to monitored. If multiple requests come in quickly, the system waits 30 seconds between each new request and checks at the end if all the requested albums are now monitored. It’s a very basic and somewhat crude solution, but it works. Without this process, I was encountering errors and unmonitored albums when users spammed requests. It can definitely be improved in the future, but for now, I prioritized getting a stable and reliable solution in place quickly.

On my end, I’m using it in production with all my users, and so far, I haven’t encountered any major bugs or errors causing server crashes. I’ll continue monitoring the logs. 👀 🔥

0-Pierre avatar Jan 14 '25 11:01 0-Pierre

Hey, @0-Pierre just popping by to say awesome work! It's fun to finally be able to play around with music requests, even though it's just a preview. Sharing some additional feedback as a Plex/Jellyseerr user:

  • No music library(s) syncing - in the initial setup process, only the movie and series type libraries are visible making it impossible to sync what music is currently available with Jellyseerr
  • Requesting an album by a monitored artist throws an error in Lidarr:

LidarrErrorPipeline Invalid request Validation failed: -- ForeignAlbumId: This album has already been added. LidarrErrorPipeline FluentValidation.ValidationException: Validation failed: -- ForeignAlbumId: This album has already been added.: Validation failed: -- ForeignAlbumId: This album has already been added.

Unmonitoring either the artist/album and making the request again results with the same error. The only work around that I've found so far is to remove the artist altogether from Lidarr and request it again. That way, it get added to Lidarr the way you described above; however, the following is observed:

  • Artist is unmonitored
  • Album is monitored
  • Download starts
  • Download is stuck at importing and after roughly 2-3 min requires manual import

Additionally, the artist gets added to Lidarr without a metadata profile; unsure if that is expected behavior. For context, I am running everything on Windows 11 and have tried using both the official Lidarr service as well as with ghcr.io/hotio/lidarr:pr-plugins, both experiencing the same.

Once again, thanks for putting in the work and time to make this!

gssariev avatar Jan 14 '25 16:01 gssariev

Hey, @0-Pierre just popping by to say awesome work! It's fun to finally be able to play around with music requests, even though it's just a preview. Sharing some additional feedback as a Plex/Jellyseerr user:

  • No music library(s) syncing - in the initial setup process, only the movie and series type libraries are visible making it impossible to sync what music is currently available with Jellyseerr
  • Requesting an album by a monitored artist throws an error in Lidarr:

LidarrErrorPipeline Invalid request Validation failed: -- ForeignAlbumId: This album has already been added. LidarrErrorPipeline FluentValidation.ValidationException: Validation failed: -- ForeignAlbumId: This album has already been added.: Validation failed: -- ForeignAlbumId: This album has already been added.

Unmonitoring either the artist/album and making the request again results with the same error. The only work around that I've found so far is to remove the artist altogether from Lidarr and request it again. That way, it get added to Lidarr the way you described above; however, the following is observed:

  • Artist is unmonitored
  • Album is monitored
  • Download starts
  • Download is stuck at importing and after roughly 2-3 min requires manual import

Additionally, the artist gets added to Lidarr without a metadata profile; unsure if that is expected behavior. For context, I am running everything on Windows 11 and have tried using both the official Lidarr service as well as with ghcr.io/hotio/lidarr:pr-plugins, both experiencing the same.

Once again, thanks for putting in the work and time to make this!

@Gssariev Thank you for your tests! I actually haven't tested it in a Plex setup situation yet. I'll review all the issues you reported and will get back with a commit to fix them. 😉

0-Pierre avatar Jan 14 '25 19:01 0-Pierre

@0-Pierre great work so far, I was hoping one day we would get this added. Some items I am seeing so far;

Running in docker with emby Settings -> Emby -> Emby libraries, music library missing Manage music /Manage Album showing [object Object] instead of Artist clearing data from lidarr only available after going to requests. 2025-01-14T23:01:34.912Z [error][Media]: [Lidarr] Failed to remove album: 404 Not Found

"play on" not showing for available media player

mr2toyou avatar Jan 14 '25 20:01 mr2toyou

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

github-actions[bot] avatar Jan 15 '25 16:01 github-actions[bot]

You should only have the en.json using the pnpm i18n:extract for the i18n locales. The rest of the translation strings should be updated through weblate.

Could you remove those translation files and rebase?

Sure, i'm fixing all the requested issues and I'll take care of that 👍

0-Pierre avatar Jan 15 '25 19:01 0-Pierre

@Gssariev Hello! I'll be pushing a new commit that resolves all the Plex-related issues. However, I'm having trouble reproducing your other issues, as everything seems to be working perfectly on my end — requests are sent and downloaded properly. I even set up a Plex test server, and everything is running smoothly with the same Docker version you're using.

I've also implemented proper mapping for the Plex music library since Plex doesn't handle it like other libraries. They rely solely on release IDs, so we’re now mapping those to release-group IDs to ensure the database gets the correct entries. Gotta love Jellyfin for handling things more consistently in this area, lol.

Let me know if you encounter any additional issues!

image image image

0-Pierre avatar Jan 15 '25 20:01 0-Pierre

@gssariev Hello! I'll be pushing a new commit that resolves all the Plex-related issues. However, I'm having trouble reproducing your other issues, as everything seems to be working perfectly on my end — requests are sent and downloaded properly. I even set up a Plex test server, and everything is running smoothly with the same Docker version you're using.

I've also implemented proper mapping for the Plex music library since Plex doesn't handle it like other libraries. They rely solely on release IDs, so we’re now mapping those to release-group IDs to ensure the database gets the correct entries. Gotta love Jellyfin for handling things more consistently in this area, lol.

Let me know if you encounter any additional issues!

image image image

Excited to test it out and will report any finding once available! The Lidarr issue could also be something on my end...

gssariev avatar Jan 15 '25 20:01 gssariev

@0-Pierre I built the image from your repository (unsure if that's the best approach), and I can confirm that my Music library does get detected and everything is being synced up, and is reflected in Jellyseerr. From the logs, I can also see that it correctly identifies some of my music as not having a MusicBrainz ID, which I expected as that music doesn't have an entry in their database - all good here.

When it comes to Lidarr...it's a bit of a mixed bag and I think the issues mainly stem from the ghcr.io/hotio/lidarr:pr-plugins version I am running with https://github.com/TrevTV/Lidarr.Plugin.Tidal plugin. The request go through if the artist is not present in the Lidarr database and take a long time to get imported (like maybe 5+ min); however, no download is initiated for albums of artists that are already being monitored/are in the Lidarr database. If I manually trigger the search from within Lidarr, everything runs smoothly and music gets imported quickly.

Running the official Lidarr service, with a bunch of indexers, seems to work better and is more consistent with what you described (initial delay of X seconds, unmonitored album, etc.); however, if the album requested is already monitored, the request screen gets stuck in a loop, but I get the same message as you in the logs and nothing else:

2025-01-16 18:20:33 jellyseerr-lidarr | 2025-01-16T17:20:33.898Z [info][Notifications]: Sending notification(s) for MEDIA_AUTO_APPROVED {"subject":"Unsainted (2019)"} 2025-01-16 18:20:33 jellyseerr-lidarr | 2025-01-16T17:20:33.985Z [info][Lidarr]: Album already exists in Lidarr, monitoring it in Lidarr {"albumTitle":"Unsainted","mbId":"924ff4f6-f0b9-4cfe-ab28-cb80a164e440"}

Ultimately, the request doesn't go through in that case. If I restart the container I can see it displayed as 'pending'.

gssariev avatar Jan 16 '25 17:01 gssariev

Hi @0-Pierre. Thanks for you great work!

I've just set it up, but all my requests are failing with the error "Metadata profile does not exist": 2025-01-16T17:17:45.652Z [error][Lidarr]: Failed to add album to Lidarr {"options":{"mbId":"a3159fb4-2ad3-4678-ba51-36fbc8b8430d","foreignAlbumId":"a3159fb4-2ad3-4678-ba51-36fbc8b8430d","title":"From Zero","qualityProfileId":4,"profileId":4,"metadataProfileId":4,"rootFolderPath":"/music","monitored":false,"tags":[],"searchNow":false,"images":[{"url":"/MediaCoverProxy/18914127a38bfd109faaebb359c3ce62836629c03ec646ee9802a0decbb85c10/from-zero-66db0c50877a9.jpg","coverType":"cover","extension":".jpg","remoteUrl":"http://assets.fanart.tv/fanart/music/f59c5520-5f46-4d2c-b2c4-822eabf53419/albumcover/from-zero-66db0c50877a9.jpg"},{"url":"/MediaCoverProxy/ee31fb45ac89c03626fafcd1c36de64bb5e78c7279e5851651c9e6b27ee067b1/40742283189-1200.jpg","coverType":"disc","extension":".jpg","remoteUrl":"https://imagecache.lidarr.audio/v1/caa/4a32e308-fbd4-4643-a419-ee90a2ee5b40/40742283189-1200.jpg"}],"addOptions":{"monitor":"none","monitored":false,"searchForMissingAlbums":false},"artist":{"foreignArtistId":"f59c5520-5f46-4d2c-b2c4-822eabf53419","artistName":"Linkin Park","qualityProfileId":4,"metadataProfileId":4,"rootFolderPath":"/music","monitored":false,"monitorNewItems":"none"}},"errorMessage":"400 Bad Request: [\n {\n \"propertyName\": \"Artist.MetadataProfileId\",\n \"errorMessage\": \"Metadata profile does not exist\",\n \"attemptedValue\": 4,\n \"severity\": \"error\",\n \"errorCode\": \"MetadataProfileExistsValidator\",\n \"formattedMessageArguments\": [],\n \"formattedMessagePlaceholderValues\": {\n \"propertyName\": \"Artist. Metadata Profile Id\",\n \"propertyValue\": 4\n }\n }\n]"}

As I've just setup lidarr and I'm new to it, it might could just be a configuration issue within lidarr.

Edit: It works, when I select the quality profile "Any", but not with any other profile.

flo1t avatar Jan 16 '25 17:01 flo1t

Issues and Fix Tracking 🎯

  • [x] Add a "Play On" button for Plex and Emby servers.
  • [x] Fix the hover style for cards that do not have images.
  • [x] Add detection and support for Emby music libraries.
  • [x] Manage slider not showing the requester and enable the ability to delete albums in Lidarr.
  • [x] Resolve the 404 error thrown when attempting to delete an album in Lidarr.
  • [x] Fix emails not being sent for music content when they become available.
  • [x] Fix the issue where, if a user requests a lot of music too fast, only the first album is monitored.
  • [x] Fix SlideOver menu not showing artist name.
  • [x] Locale creation via pnpm i18n:extract.
  • [x] Rebase to ensure that we get latest commit and fix code erased.

Known issues that I will not fix myself, help would be needed. ✋

  • Plex Recently Added Scan not fetching music, but complete one does. (Maybe emby facing same issue)
  • We can't cancel the music library fetching when started on each media server, cancel button doesn't do anythings.

0-Pierre avatar Jan 16 '25 17:01 0-Pierre

I tried integration with Lidarr. There are several questions and suggestions.

  1. Is it possible to move from the request card or album card to the artist? 1 2.On some albums, the query button is constantly visible. 2
  2. Is it possible to request the entire artist instead of the album?
  3. Make it possible to switch from the album to the artist.

3

Druidblack avatar Jan 16 '25 18:01 Druidblack

@0-Pierre any update with the fix of the merge issues? It seems quite critical to us, and we'd like it to be fixed as soon as possible before things become even more complicated.

If you're unsure on how to proceed, it's ok we can help and do it ourselves, but please let us know

gauthier-th avatar Jan 18 '25 22:01 gauthier-th

@gauthier-th

I'd be glad if you could take care of this, as you're much more comfortable with the rebase process than I am. 🤝

Should I stop commiting for the moment ?

0-Pierre avatar Jan 19 '25 08:01 0-Pierre

@gauthier-th

I'd be glad if you could take care of this, as you're much more comfortable with the rebase process than I am. 🤝

Should I stop commiting for the moment ?

Yes please. I'll try to do it by 1/2 days.

gauthier-th avatar Jan 19 '25 21:01 gauthier-th

Hi All, one of the Servarr devs here (we maintain Lidarr and it's backend). The Lidarr API exists solely for Lidarr's use and is aimed at it. We run a MusicBrainz mirror and ultimately do not test any changes with any products that are not Lidarr nor do we expect to support alternative products.

As much as I appreciate what you are trying to do, there has been no conversation with ourselves surroudning the user of our API and our general expectation would be that Jellyseerr would run a similar service themselves to support this instead of using our API as then changes to it can be tested/managed for compatibility with Jellyseerr.

RobinDadswell avatar Jan 20 '25 09:01 RobinDadswell

Hi All, one of the Servarr devs here (we maintain Lidarr and it's backend). The Lidarr API exists solely for Lidarr's use and is aimed at it. We run a MusicBrainz mirror and ultimately do not test any changes with any products that are not Lidarr nor do we expect to support alternative products.

As much as I appreciate what you are trying to do, there has been no conversation with ourselves surroudning the user of our API and our general expectation would be that Jellyseerr would run a similar service themselves to support this instead of using our API as then changes to it can be tested/managed for compatibility with Jellyseerr.

Hi, sorry for the use of your API. We (maintainers) didn't catch that from this PR as we have not yet reviewed this and it wasn't our intention to use something we shouldn't have (we thank you for the confirmation).

However, would you be willing to discuss with us about the use of this API? If not, that's fine, thanks for your awesome work on Servarr and Lidarr :slightly_smiling_face:

gauthier-th avatar Jan 20 '25 13:01 gauthier-th

Hi All, one of the Servarr devs here (we maintain Lidarr and it's backend). The Lidarr API exists solely for Lidarr's use and is aimed at it. We run a MusicBrainz mirror and ultimately do not test any changes with any products that are not Lidarr nor do we expect to support alternative products. As much as I appreciate what you are trying to do, there has been no conversation with ourselves surroudning the user of our API and our general expectation would be that Jellyseerr would run a similar service themselves to support this instead of using our API as then changes to it can be tested/managed for compatibility with Jellyseerr.

Hi, sorry for the use of your API. We (maintainers) didn't catch that from this PR as we have not yet reviewed this and it wasn't our intention to use something we shouldn't have (we thank you for the confirmation).

However, would you be willing to discuss with us about the use of this API? If not, that's fine, thanks for your awesome work on Servarr and Lidarr 🙂

Our options are quite limited. The MusicBrainz API is too slow and has a low rate limit, which prevents it from delivering a smooth experience like TMDB. A possible solution would be to set up a Jellyseerr mirror API, similar to Lidarr's approach. This mirror API could prefetch data, populate the database with CCA images, and retrieve artist images from Fanart.tv using a global API key and maybe the missing one with LastFM with a global key for the API. This approach could offer additional benefits, such as more detailed data, preloaded charts, and an overall enhanced experience.

0-Pierre avatar Jan 20 '25 17:01 0-Pierre

It's always possible to self-host a music brainz instance, they provide a full setup guide and everything. Issue is that it requires comparably potent hardware.

Shadowghost avatar Jan 20 '25 17:01 Shadowghost

It's always possible to self-host a music brainz instance, they provide a full setup guide and everything. Issue is that it requires comparably potent hardware.

Yep, I started to setup a mirror on a VPS I don't really use, but the issue is the hardware requirements that are quite high, especially if we want something fast enough for thousands of users. The costs of running such an instance would be quite high.

gauthier-th avatar Jan 20 '25 18:01 gauthier-th

Using MusicBrainz would work but take more time to gather all the data, which I think is an alright compromise, isn't it? Even it it takes a couple of days to scan a complete library, it shouldn't be a big deal.

jvoisin avatar Jan 20 '25 18:01 jvoisin

I think there are multiple aproaches to attack this problem (some can be combined)

  1. Only us musicbrain
  2. Let the user choose an MB-instance of their own
  3. Create a donation based api
  4. Create a premium service
  5. Create a complete new project for cloning the mb database with the possibility of crowdpower (every user can donate compute power, ram and storage [e.g. torrent or blockchain storage])

Other ideas?

DunklerPhoenix avatar Jan 20 '25 19:01 DunklerPhoenix

Why not use arl deezer? and other alternative sources for information. The MB database does not have the same content as streaming services. And their moderation in 7 days is depressing.

Druidblack avatar Jan 20 '25 19:01 Druidblack

I can easily adjust the API to use MusicBrainz instead, but this would require all images to be fetched from CCA for covers. Since CCA servers are quite slow, these images will need to be cached. Additionally, we would need to use a global API key from Fanart (similar to the TMDB key) to retrieve artist covers and images. I will create a new branch lidarr-mb to make tests.

0-Pierre avatar Jan 20 '25 19:01 0-Pierre

Why not use arl deezer? and other alternative sources for information. The MB database does not have the same content as streaming services. And their moderation in 7 days is depressing.

I've already considered this, but the main issue is that Lidarr relies on unique mbIds, while Deezer does not, which would inevitably cause mismatches over time. To ensure accuracy in media requests without manually mapping each item to the correct ID and to avoid running into rate limits we must adhere to the source app's identification system. The only viable solution would be a version of Lidarr built entirely around using Deezer as its indexer.

0-Pierre avatar Jan 20 '25 19:01 0-Pierre