Spotify API seemingly had a breaking change a few days ago

[ppege]

Describe the bug A clear and concise description of what the bug is.

Several of Spotify's API endpoints are now returning errors to the requests rspotify is issuing it. For example:

thread 'main' panicked at src/main.rs:133:55:
called `Result::unwrap()` on an `Err` value: ParseJson(Error("invalid type: null, expected struct SimplifiedPlaylist", l
ine: 1, column: 4780))

This seems to be happening after this new update that just came out for Spotify's web API. More context in this issue.

To Reproduce Steps to reproduce the behavior: For me, this error appears when I await the AuthCodeSpotify client's current_user_playlists method.

Expected behavior Usually this method would simply get the playlists of the authenticated user. Auth does seem to work currently though.

Log/Output data If applicable, add log data or output data to help explain your problem.

Additional context Add any other context about the problem here.

[buzzneon]

I'm seeing this too - specifically a result of calling current_user_playlists_manual, the API is return null entries in the array:

{
  "href": "https://api.spotify.com/v1/users/buzzneon/playlists?offset=0&limit=50",
  "limit": 50,
  "next": null,
  "offset": 0,
  "previous": null,
  "total": 10,
  "items": [
    { ... },
    { ... },
    null,
    null,
    { ... },
    { ... },
    { ... },
    null,
    { ... },
    null
  ]
}

(Note, I truncated the output - omitting the valid playlists). It appears that the null playlists are the ones which Spotify generated for me, specifically the "Your top songs 2023". A "simple" fix for this would be to have current_user_playlists and current_user_playlists_manual return Page<Option<SimplifiedPlaylist>>, though that's a breaking change which isn't terribly appealing. Maybe have those function internally filter out the null entries? I like this from the perspective of a user of this library; however there will be a performance hit (which doesn't bother me, but I can't speak for everyone!).

[buzzneon]

I mocked something up, which I have working locally, in rspotify-modal/src/page.rs you can define a way to build a Page of firm values from a Page of optional ones. This will consume the page of optional values, and move all Some<T> items from one page to the other (no copying or cloning). I tried to make a PR, but I don't think I have access.

impl<T> From<Page<Option<T>>> for Page<T> {
    fn from(mut value: Page<Option<T>>) -> Self {
        let items = value.items
            .iter_mut().filter_map(|item| item.take() )
            .collect();
        Self {
            href: value.href,
            items,
            limit: value.limit,
            next: value.next,
            offset: value.offset,
            previous: value.previous,
            total: value.total
        }
    }
}

I wrote a test for this too:

    #[test]
    fn can_create_page_from_page_options() {

        // Test a vector with all Some
        let page: Page<i32> = Page::from(Page {
            items: vec![Some(1), Some(2), Some(3), Some(4), Some(5)],
            ..Default::default()
        });
        assert_eq!(page.items, vec![1, 2, 3, 4, 5]);

        // Test a vector with all None
        let page: Page<i32> = Page::from(Page {
            items: vec![None, None, None, None],
            ..Default::default()
        });
        assert_eq!(page.items, Vec::<i32>::new());

        // Test a vector with mixed items.
        let page: Page<i32> = Page::from(Page {
            items: vec![Some(1), None, None, Some(4), Some(5), None],
            ..Default::default()
        });
        assert_eq!(page.items, vec![1, 4, 5]);
    }

And I hooked it up to current_user_playlists_manual in src/clients/oauth.rs:

    async fn current_user_playlists_manual(
        &self,
        limit: Option<u32>,
        offset: Option<u32>,
    ) -> ClientResult<Page<SimplifiedPlaylist>> {
        let limit = limit.map(|s| s.to_string());
        let offset = offset.map(|s| s.to_string());
        let params = build_map([("limit", limit.as_deref()), ("offset", offset.as_deref())]);

        let result = self.api_get("me/playlists", &params).await?;
        let page: Page<Option<SimplifiedPlaylist>> = convert_result(&result)?;
        Ok(Page::from(page))
    }

I imagine the same trait could be implemented for CursorBasedPage too.

I only tested this on current_user_playlists_manual since that was the only API call I am making which was failing, and I can confirm that it now works, without breaking compatibility.

[ramsayleung]

Thanks for your report:

It appears that the null playlists are the ones which Spotify generated for me, specifically the "Your top songs 2023".

Do you have any clue why would the Spotify generate fnull playlists for your "Your top songs 2023", how does it looks like in your player?

[CameronSilverTXI]

In the Spotify app, the playlists appear without issues .. I assume returning null here is part of Spotify's supposed "security improvements" behind the API changes.

[dzubybb]

Is it possible that those "security improvements" are the reason that i'm getting null in context from current_playback, even when other device is playing some playlist ?

[ramsayleung]

This problem should be fixed by this PR https://github.com/ramsayleung/rspotify/pull/526, v0.15.0 is out, which should address this issue.