popyt icon indicating copy to clipboard operation
popyt copied to clipboard

Fetchvideos max number of elements

Open 2colours opened this issue 3 years ago • 8 comments

Hello When I tried to call fetchVideos with a number greater than 50, the library throws an error. This seems a bit harsh given that 100 still seems like a reasonable size for playlists. Why does this limitation exist? Is there a workaround to indeed do fetch all the videos of the playlist, at least up to a greater limit? This is really a serious limitation for me at the moment, as I'm using the library for developing a musicbot on Discord. I hope there can be a solution. Thank you for a response.

2colours avatar Jul 31 '21 11:07 2colours

The reason this limitation exists is because the YouTube Data API limits each request for playlist items to 50 videos. It provides two tokens in each request, the previous page token and the next page token. When requesting paginated items, one can provide a page token to be given the next page of items rather than just the first 50.

popyt does not currently have the functionality to work with pages with its public functions like this, but I am more than happy to implement this.

Personal notes on my progress (will commit and publish when complete):

  • [x] Return prev and next page tokens in paginated responses
  • [x] Add starting pageToken parameter to paginated methods
  • [x] Add pages parameter to paginated methods to allow for simple multi-page requests
  • [ ] Update entity methods to include this functionality
  • [ ] Update documentation
  • [ ] Add tests for all this

brandonbothell avatar Aug 02 '21 01:08 brandonbothell

I came for this same issue. I think there's a discrepancy in the docs for getPlaylistItems which says the following:

The maximum amount of videos to get from the playlist. If <=0, returns all videos in the playlist.

In the meantime I think you should update this to "returns the first 50" rather than "returns all"

phazonoverload avatar Aug 02 '21 10:08 phazonoverload

I'm very thankful for your work on this library so no pressure to commit, but I'm wondering how long you think it will take to implement this? Just planning whether I'll need a workaround or if I should wait 😄

phazonoverload avatar Aug 02 '21 10:08 phazonoverload

I came for this same issue. I think there's a discrepancy in the docs for getPlaylistItems which says the following:

The maximum amount of videos to get from the playlist. If <=0, returns all videos in the playlist.

In the meantime I think you should update this to "returns the first 50" rather than "returns all"

@phazonoverload The library does actually fetch every page of the playlist's items if maxResults is <= 0. There's just not a way for the library to fetch more than 50 but less than all of the items, which is what I am implementing to fix this issue. This should max be committed and released by Thursday.

brandonbothell avatar Aug 02 '21 15:08 brandonbothell

Makes sense - semantically I expected this method to handle the pagination and return all items. Thanks again, and look forward t seeing when you make this new release <3

phazonoverload avatar Aug 02 '21 15:08 phazonoverload

Hey @jasonhaxstuff - hope you're having a wonderful start to your month. Just wanted to check in about this issue and when you might expect this functionality to land? Thanks again

phazonoverload avatar Sep 02 '21 16:09 phazonoverload

Hey @jasonhaxstuff - hope you're having a wonderful start to your month. Just wanted to check in about this issue and when you might expect this functionality to land? Thanks again

Howdy @phazonoverload, sorry I've gone dead on you, but I recently started college and my focus has been on that. I should have some free time this weekend to work on it. For now I have created a pull request that you can contribute to if you'd like. Thanks for your interest in my library!

brandonbothell avatar Sep 02 '21 16:09 brandonbothell

100% understood and absolutely the right priority. I'll take a look at the PR and if there's anything I can do to help

phazonoverload avatar Sep 02 '21 18:09 phazonoverload

Fixed in v6.1.1. With the new options on paginated methods, you can choose how many items are included per page as well as how many pages to fetch.

brandonbothell avatar Jun 21 '23 02:06 brandonbothell