Load data in chunck instead of making a single request

[Extarys]

Describe The Bug

Opening any library make a request that asks all the data. (All artists, all movies...) The UI would feel more responsive if we only request X items at a time. It would also reduce server load and data transfered for most use cases.

Steps To Reproduce

1. Press F12 and go to the Network tab
2. Go to your music library (default view to Album)
3. Check the time and size of the request for the music library
4. Go to Artists view
5. You have loaded 100 albums, 500 albums? waited a little while and transfered 1MB/2MB+ of useless data so you can just browse by artists

Expected Behavior

Load a certain amount of data, display them then slowly request more data in the background.

Faster UI response, less useless data being transfered.

Screenshots

System (please complete the following information):

• Platform: [e.g. Linux, Windows, iPhone, Tizen]
• Browser: [e.g. Firefox, Chrome, Safari]
• Jellyfin Version: [e.g. 10.6.0]

Additional Context

I'm using Jellyfin on a RPi 4. The experience is smoother with 10.5+ if I remember this properly, but before that most, RPi users had a hard time browsing the library on a RPi. Big libraries feel slower and slower, with all the images that need to be loaded on top of the data on a cellular network too, it starts to add up quickly.

In comparison, Navidrome (which only support/do music) feels 10x times as fast as Jellyfin, but they do use pagination. :cry: :no_good_man:

[stale[bot]]

Issues go stale after 60 days of inactivity. Mark the issue as fresh by adding a comment or commit. Stale issues close after an additional 14 days of inactivity. If this issue is safe to close now please do so. If you have any questions you can reach us on Matrix or Social Media.

[Extarys]

Bad bot

[stale[bot]]

Issues go stale after 60 days of inactivity. Mark the issue as fresh by adding a comment or commit. Stale issues close after an additional 14 days of inactivity. If this issue is safe to close now please do so. If you have any questions you can reach us on Matrix or Social Media.

[HEPOSHEIKKI]

This would make the user experience much more enjoyable. +1

[ZachMyers3]

I was glancing at the SDK to see if this would be possible. I'm really looking for three things in the itemsAPI.

1. Total # of items in given query (cannot find)
2. Limit # of items coming back in a given query here
3. Offset # of items for next page to load here

I think even without the total # of items it's possible to use v-intersect to handle the event to fetch the next page once a user scrolls down enough and adjust the component(s) to handle this accordingly.

Without a total it might be a bit hacky to determine the bottom of the page, but still possible.

Thoughts?

[ferferga]

@ZachMyers3 VIntersect can't be used, since we have our custom virtual scroller (based on https://github.com/rocwang/Vue-virtual-scroll-grid, which has this feature of infinite scrolling while fetching data). It wouldn't be too difficult to implement.

In my opinion, our problem is that the server is slow and we would be fine by simply fetching all the libraries in the background after a user has logged in, so all the data is populated once clicked.

When the library is accessed, it can be refetched and reactively updated.

PS: And yes, there are Count methods in API. However, the way the server is built basically fetches everything and then limits, which is inefficient. Again this is another reason why I think we're better off as well by doing a single transaction and that's it.

[ZachMyers3]

I think I'm understanding, thanks for the feedback.

You're thinking it's probably better to fetch all the item collections asynchronously, probably the same time the list of item collections is fetched for the drop-down?

I can try that out instead, that's easier to implement anyway!

[ferferga]

@ZachMyers3 sorry, which drop down you mean? 😅

[ZachMyers3]

Here is where I'm thinking. Would this be too heavy, or better stored in another store?

[ZachMyers3]

I've made a PR with what I was thinking, whenever you have some time let me know your thoughts.

[ferferga]

@ZachMyers3 @HEPOSHEIKKI @Extarys Fixed in a best-effort way in #2201

As you can see, even with 50 items the server struggles, but that's nothing I can do in my end, so I'd say this is a good solution for now and this issue can be closed when the linked PR is merged.

Please test and provide feedback.

[Extarys]

@ferferga You are the best.

Should I/someone open an issue for jellyfin-server to add some kind of real pagination/limit? I'm astonished that the server still load everything then limit the number of entries for a given request.

[HEPOSHEIKKI]

@ZachMyers3 @HEPOSHEIKKI @Extarys Fixed in a best-effort way in #2201

As you can see, even with 50 items the server struggles, but that's nothing I can do in my end, so I'd say this is a good solution for now and this issue can be closed when the linked PR is merged.

Please test and provide feedback.

Tested and confirmed working with no apparent problems. I think this issue can be marked as solved after merging.

Thank you!

[ferferga]

@Extarys Not needed, but thank you very much for your interest!

We've been discussing this already in jellyfin-meta and internally, we all know the API and the DB sucks 😣.

Until both things are fixed, this is still going to be a problem.

In the meantime, all I can do are some little tricks (like this one) to ensure the experience is a bit better and that virtual scrolling is performant enough.