JSON parse error because Spotify returns wrong data

[davidemarcoli]

Describe the bug A clear and concise description of what the bug is.

The spotify returns wrong data which isn't being handled and an exception is thrown.

To Reproduce Steps to reproduce the behavior:

1. Get this playlist from the api: 0hnM986qSkpI9YSFwpl96E

Expected behavior A clear and concise description of what you expected to happen.

It can parse the response without exception

Log/Output data If applicable, add log data or output data to help explain your problem.

data did not match any variant of untagged enum PlayableItem at line 1 column 86707

Additional context Add any other context about the problem here.

Playlist which it fails for: https://open.spotify.com/playlist/0hnM986qSkpI9YSFwpl96E/

Example of an item: https://open.spotify.com/episode/60XIexbGJJYcc2R5idt8p4 The api returns track=true and episode=false, which obviously is not true

[ramsayleung]

Which API you were trying to call to get the playlist?

[davidemarcoli]

I used the playlist_items_manual function. Meaning the following spotify endpoint: https://api.spotify.com/v1/playlists/0hnM986qSkpI9YSFwpl96E/tracks

[ramsayleung]

This is my unit test, it's passed, I can't re-produce this problem:

#[maybe_async::test(
    feature = "__sync",
    async(all(feature = "__async", not(target_arch = "wasm32")), tokio::test),
    async(all(feature = "__async", target_arch = "wasm32"), wasm_bindgen_test)
)]
async fn test_playlist_deserialize() {
    let playlist_id = PlaylistId::from_id("0hnM986qSkpI9YSFwpl96E").unwrap();
    let result = creds_client().await.playlist_items_manual(playlist_id, None, None, None, None).await;
    println!("result: {:#?}", result);
}

Could you run this test, and post your output?

cargo test --no-default-features --features=env-file,client-ureq,ureq-rustls-tls -- --nocapture > /tmp/cargo-out.txt

[DrunkenToast]

Running into the same issue, although not with the playlist above, instead with this one: 4MAcblnU4nwtIZ267YI24a

Response: Err(
    ParseJson(
        Error("data did not match any variant of untagged enum PlayableItem", line: 1, column: 52188),
    ),
)

The output of the request is this: out.txt

Seems to be on the video thumbnail? One of the tracks is a podcast episode. Not sure where the issue lies but hope it helps.

[ramsayleung]

The root cause should be the "type": "REALPODCASTNOTMUSIC123", Spotify rollouts a new type again.

It's a recurring problem I encounter again and again, I couldn't count how many times Spotify rollout a new field or new variant without properly updating their CHANGELOG, and user will create an issue to report the JSON deserialization error.

I am proposing a more robust way to minimize the impact to this library.

[DrunkenToast]

That seems like the most sensible solution. Thanks for the hard work. The Spotify API seems like quite a mess :)

[ramsayleung]

I eventually figure out the root cause of this JSON error, I can reproduce the problem with this test:

#[test]
#[wasm_bindgen_test]
fn test_deserialization_playlist_item_with_malformed_episodes() {
    // To fix https://github.com/ramsayleung/rspotify/issues/525
    let json = r#"
{
  "href": "https://api.spotify.com/v1/playlists/4MAcblnU4nwtIZ267YI24a/tracks?offset=0&limit=100",
  "items": [
    {
      "added_at": "2024-12-14T13:23:53Z",
      "added_by": {
        "external_urls": {
          "spotify": "https://open.spotify.com/user/31rhaare4k4nkr5bngqyiiz4bq6a"
        },
        "href": "https://api.spotify.com/v1/users/31rhaare4k4nkr5bngqyiiz4bq6a",
        "id": "31rhaare4k4nkr5bngqyiiz4bq6a",
        "type": "user",
        "uri": "spotify:user:31rhaare4k4nkr5bngqyiiz4bq6a"
      },
      "is_local": false,
      "primary_color": null,
      "track": {
        "preview_url": "https://podz-content.spotifycdn.com/audio/clips/3pEQnFvjtRVnJURokrIEaH/clip_46844_106844.mp3",
        "available_markets": ["AD"],
        "explicit": false,
        "type": "episode",
        "episode": false,
        "track": true,
        "album": {
          "available_markets": ["AD"],
          "type": "show",
          "album_type": "compilation",
          "href": "https://api.spotify.com/v1/shows/4C53FwIBO9tEhJ0dkMN3GN",
          "id": "4C53FwIBO9tEhJ0dkMN3GN",
          "images": [
            {
              "height": 64,
              "url": "https://i.scdn.co/image/ab6765630000f68dc5087ef7f8e3caa4fd6e6bcd",
              "width": 64
            }
          ],
          "name": "REALPODCASTNOTMUSIC123",
          "release_date": null,
          "release_date_precision": null,
          "uri": "spotify:show:4C53FwIBO9tEhJ0dkMN3GN",
          "artists": [
            {
              "external_urls": {
                "spotify": "https://open.spotify.com/show/4C53FwIBO9tEhJ0dkMN3GN"
              },
              "href": "https://api.spotify.com/v1/shows/4C53FwIBO9tEhJ0dkMN3GN",
              "id": "4C53FwIBO9tEhJ0dkMN3GN",
              "name": null,
              "type": "REALPODCASTNOTMUSIC123",
              "uri": "spotify:show:4C53FwIBO9tEhJ0dkMN3GN"
            }
          ],
          "external_urls": {
            "spotify": "https://open.spotify.com/album/4C53FwIBO9tEhJ0dkMN3GN"
          },
          "total_tracks": 1
        },
        "artists": [
          {
            "external_urls": {
              "spotify": "https://open.spotify.com/show/4C53FwIBO9tEhJ0dkMN3GN"
            },
            "href": "https://api.spotify.com/v1/shows/4C53FwIBO9tEhJ0dkMN3GN",
            "id": "4C53FwIBO9tEhJ0dkMN3GN",
            "name": null,
            "type": "REALPODCASTNOTMUSIC123",
            "uri": "spotify:show:4C53FwIBO9tEhJ0dkMN3GN"
          }
        ],
        "disc_number": 0,
        "track_number": 0,
        "duration_ms": 119257,
        "external_ids": {
          "spotify": "https://open.spotify.com/episode/4IBGQd8aV4j6WGqGOjdJmE"
        },
        "external_urls": {
          "spotify": "https://open.spotify.com/episode/4IBGQd8aV4j6WGqGOjdJmE"
        },
        "href": "https://api.spotify.com/v1/episodes/4IBGQd8aV4j6WGqGOjdJmE",
        "id": "4IBGQd8aV4j6WGqGOjdJmE",
        "name": "ITS BEGINNING TO LOOK A LOT LIKE CHRISTMAS (Mac Demarco)",
        "popularity": 0,
        "uri": "spotify:episode:4IBGQd8aV4j6WGqGOjdJmE",
        "is_local": false
      },
      "video_thumbnail": {
        "url": null
      }
    }
  ],
  "limit": 100,
  "next": null,
  "offset": 0,
  "previous": null,
  "total": 17
}
"#;
    let page: Page<PlaylistItem> = deserialize(json);
    assert_eq!(page.total, 1);
}

"track": {
  "type": "episode",     // <- This says it's an episode
  "episode": false,      // <- But this contradicts it
  "track": true,         // <- And this says it's a track
}

The problem is that the PlayableItem enum expects either a Track or Episode, but the JSON contains a hybrid object that has characteristics of both.

But based on API doc of get playlist item:

Information about the track or episode. Will be one of the following:

It's pretty frustrating.

[ramsayleung]

The patch has been merged into the main branch, currently you could consume the main branch to resolve the JSON error until I release a new version.

[DrunkenToast]

Hahaha, hilarious! Beautiful API design and documentation. Thank you again for resolving this so quickly! I was using reqwest manually as temporary solution but I am happy to switch back to this library. :)

[hrkfdn]

Hey @ramsayleung, thanks a lot for fixing this. I'm also getting user reports that look like this. Any chance of a patch release soon? :)

[ramsayleung]

Request received and processed, the new version of this patch is out :)

https://crates.io/crates/rspotify

[hrkfdn]

Thanks so much! :)