TeamNewPipe
[SoundCloud] Fix soundcloud regression of expiring HLS streams after 5mins (APK DL in description)
What is it?
- [x] Bugfix (user facing)
- [ ] Feature (user facing)
- [x] Codebase improvement (dev facing)
- [ ] Meta improvement to the project (dev facing)
Description of the changes in your PR
- Add refreshing of expired SoundCloud HLS playlists
- Bump ExoPlayer to latest version
- Add LoggingHttpDataSource to log ExoPlayer Http requests for non-YouTube streams
- Hook up Logcat to ExtractorLogger
- Allow comment suppression in checkstyle
- Improve logging in various places
- Small refactors in various places
Fixes the following issue(s)
- Fixes #12109
- MIGHT be a fix for #9925
Relies on the following changes (PLEASE REVIEW THIS FIRST)
- https://github.com/TeamNewPipe/NewPipeExtractor/pull/1325
APK testing
~The APK can be found by going to the "Checks" tab below the title. On the left pane, click on "CI", scroll down to "artifacts" and click "app" to download the zip file which contains the debug APK of this PR. You can find more info and a video demonstration on this wiki page.~ APK is here
Due diligence
- [x] I read the contribution guidelines.
Testing
Until we add tests, the simplest way to test this is to:
- Load a SoundCloud track, preferably one 3-4mins long (https://soundcloud.com/minecraftwizards/18-c418-sweden)
- Play it for 3s
- Pause it for 5mins
- Play it again for 2mins
If you get no error then it means it's working. You can also check Logcat and Ctrl+F for "refreshPlaylist" which should show up on step 4.
General rundown of the fix
As detailed in #12109, the cause of the bug is due to the HLS CDN urls expiring after 5 minutes. Previously we would only retrieve Progressive MP3 streams which basically just downloaded the whole track outright so there was no opportunity for anything to expire since the full track gets buffered way quicker than 5 minutes.
The way we extract SoundCloud tracks in SoundcloudStreamExtractor is we call the API to get the JSON information for a track. The part of the track that has information for the actual audio track is transcodings.
Here's an example JSON object for the track https://soundcloud.com/jaronsteele/as-the-world-gets-smaller:
Example track JSON
{
"artwork_url": "https://i1.sndcdn.com/artworks-000606732097-86fmxf-large.jpg",
"caption": null,
"commentable": true,
"comment_count": 92,
"created_at": "2019-10-02T16:48:07Z",
"description": "-\n1/6\nEverything in Between",
"downloadable": false,
"download_count": 0,
"duration": 139892,
"full_duration": 139923,
"embeddable_by": "all",
"genre": "Everything in Between",
"has_downloads_left": false,
"id": 690040873,
"kind": "track",
"label_name": null,
"last_modified": "2022-04-30T15:11:44Z",
"license": "all-rights-reserved",
"likes_count": 2745,
"permalink": "as-the-world-gets-smaller",
"permalink_url": "https://soundcloud.com/jaronsteele/as-the-world-gets-smaller",
"playback_count": 126750,
"public": true,
"publisher_metadata": {
"id": 690040873,
"urn": "soundcloud:tracks:690040873",
"contains_music": true
},
"purchase_title": null,
"purchase_url": null,
"release_date": null,
"reposts_count": 291,
"secret_token": null,
"sharing": "public",
"state": "finished",
"streamable": true,
"tag_list": "Jaron",
"title": "As The World Gets Smaller",
"uri": "https://api.soundcloud.com/tracks/690040873",
"urn": "soundcloud:tracks:690040873",
"user_id": 232789711,
"visuals": null,
"waveform_url": "https://wave.sndcdn.com/jPI4kiRuqQ8N_m.json",
"display_date": "2019-10-02T16:48:07Z",
"media": {
"transcodings": [
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/67cae8d9-7363-4aff-8003-2e905b315f86/stream/hls",
"preset": "aac_160k",
"duration": 139892,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mp4; codecs=\"mp4a.40.2\""
},
"quality": "sq",
"is_legacy_transcoding": false
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/afed15d4-0bf0-48ec-a591-d356b13486c2/stream/hls",
"preset": "abr_sq",
"duration": 139892,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mpegurl"
},
"quality": "sq",
"is_legacy_transcoding": false
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/588daf99-dcfc-47cc-8350-8df87d23bc12/stream/hls",
"preset": "mp3_0_0",
"duration": 139923,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mpeg"
},
"quality": "sq",
"is_legacy_transcoding": true
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/588daf99-dcfc-47cc-8350-8df87d23bc12/stream/progressive",
"preset": "mp3_0_0",
"duration": 139923,
"snipped": false,
"format": {
"protocol": "progressive",
"mime_type": "audio/mpeg"
},
"quality": "sq",
"is_legacy_transcoding": true
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/d9110040-fe52-4adb-84f0-7d672d7ab077/stream/hls",
"preset": "opus_0_0",
"duration": 139853,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/ogg; codecs=\"opus\""
},
"quality": "sq",
"is_legacy_transcoding": true
}
]
},
"station_urn": "soundcloud:system-playlists:track-stations:690040873",
"station_permalink": "track-stations:690040873",
"track_authorization": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJnZW8iOiJHQiIsInN1YiI6IiIsInJpZCI6IjhlNzIxMDhmLWRmNzctNDAwZC05OGM1LTQ0YzRhYjliZDJjOSIsImlhdCI6MTc0OTYzMjE1Mn0.d0DzxofzLlnLqiwSPEXwvuzundSHDbeLJQmYAIMbQ1U",
"monetization_model": "BLACKBOX",
"policy": "MONETIZE",
"user": {
"avatar_url": "https://i1.sndcdn.com/avatars-twmTztScdRSRjtfu-nIRvzg-large.jpg",
"first_name": "\\__*",
"followers_count": 21915,
"full_name": "\\__*",
"id": 232789711,
"kind": "user",
"last_modified": "2025-05-05T07:09:58Z",
"last_name": "",
"permalink": "jaronsteele",
"permalink_url": "https://soundcloud.com/jaronsteele",
"uri": "https://api.soundcloud.com/users/232789711",
"urn": "soundcloud:users:232789711",
"username": "jaron",
"verified": true,
"city": "",
"country_code": null,
"badges": {
"pro": false,
"creator_mid_tier": false,
"pro_unlimited": false,
"verified": true
},
"station_urn": "soundcloud:system-playlists:artist-stations:232789711",
"station_permalink": "artist-stations:232789711"
}
},
transcodings is found in track.media.transcodings
Here is the transcodings array for this track:
Example transcodings JSON
"media": {
"transcodings": [
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/67cae8d9-7363-4aff-8003-2e905b315f86/stream/hls",
"preset": "aac_160k",
"duration": 139892,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mp4; codecs=\"mp4a.40.2\""
},
"quality": "sq",
"is_legacy_transcoding": false
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/afed15d4-0bf0-48ec-a591-d356b13486c2/stream/hls",
"preset": "abr_sq",
"duration": 139892,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mpegurl"
},
"quality": "sq",
"is_legacy_transcoding": false
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/588daf99-dcfc-47cc-8350-8df87d23bc12/stream/hls",
"preset": "mp3_0_0",
"duration": 139923,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/mpeg"
},
"quality": "sq",
"is_legacy_transcoding": true
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/588daf99-dcfc-47cc-8350-8df87d23bc12/stream/progressive",
"preset": "mp3_0_0",
"duration": 139923,
"snipped": false,
"format": {
"protocol": "progressive",
"mime_type": "audio/mpeg"
},
"quality": "sq",
"is_legacy_transcoding": true
},
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/d9110040-fe52-4adb-84f0-7d672d7ab077/stream/hls",
"preset": "opus_0_0",
"duration": 139853,
"snipped": false,
"format": {
"protocol": "hls",
"mime_type": "audio/ogg; codecs=\"opus\""
},
"quality": "sq",
"is_legacy_transcoding": true
}
]
},
Each transcoding in the array has a url, and we call that endpoint to get the direct CDN stream url we need to get the actual binary data which will be played by ExoPlayer.
The CDN urls for each transcoding all have a different format for the base path of the URL, but they share some query parameters.
Example Progressive MP3 URL
https://cf-media.sndcdn.com/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLW1lZGlhLnNuZGNkbi5jb20valBJNGtpUnVxUThOLjEyOC5tcDMqIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzUwODk1ODU5fX19XX0_&Signature=dhm-WTxVYck7kCeqQpYS6LHblPqtwseUkzZ4hKjUnRnOQGwe~6hbgLMW~TE2l8QEiGlUFBMS4LsuDTH8sBCZzyjJ0vdOdHucphg9se-P8ZCUCjxVOfI16DLAMlb3KfSAkyeqZUWpuRf0Zq9AmdNRhBBKiexycruaQCYGMK~Qe9HaCCXWTYZamHSogitnif~r5ga5jeZs23FU30al6RzeKN64pwMnYdi1pnVixEykaQ5b4Zg6hLRZp~a7gqFqqyBX8PESzH2hJkV1rphNtnHIl0C1vgfxj9hltkrNhQDri~OD4e4a~nqTmnAkUFqSuCkVXWKyfIgICoMMX2~jdpoquQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
Example HLS MP3 URL
https://cf-hls-media.sndcdn.com/playlist/jPI4kiRuqQ8N.128.mp3/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL3BsYXlsaXN0L2pQSTRraVJ1cVE4Ti4xMjgubXAzL3BsYXlsaXN0Lm0zdTgqIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzUwODk0OTE4fX19XX0_&Signature=BRwoetXgqu0HndAziOtz48aKzwH1uc07ruDeAECzSKGHN5XK2s20E4hQaLR4PzCix5AZn-GKsU5Myqz~wp2uhIWHKXfyhn4aQNTBxIIREfwR9wGTNKVcSA5IGjtmjJF37uVuAkxSwPSEg54I9MB6MvftSy4P5twTLEj~x3xfX9k6mxIaqoBqMP5TuuLFRqRnIBa~PEK~IfY~SKWH5swv9ZSQgbKSlm1bznb66SI189wMes1n1Z1UxaVEXn2-PCFHBPkaH--yFd-8U4QltBbAcyLllZzpp3kIxq9BNd2ff-k8p6pilnntTerMFuzg37PQkTw8-SKcApiqoDsr-Xhmng__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
Example HLS AAC URL
https://playback.media-streaming.soundcloud.cloud/jPI4kiRuqQ8N/aac_160k/67cae8d9-7363-4aff-8003-2e905b315f86/playlist.m3u8?expires=1750894776&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbGF5YmFjay5tZWRpYS1zdHJlYW1pbmcuc291bmRjbG91ZC5jbG91ZC9qUEk0a2lSdXFROE4vYWFjXzE2MGsvNjdjYWU4ZDktNzM2My00YWZmLTgwMDMtMmU5MDViMzE1Zjg2L3BsYXlsaXN0Lm0zdTg~ZXhwaXJlcz0xNzUwODk0Nzc2IiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzUwODk0Nzc2fX19XX0_&Signature=hYW9T6gng6ixxLnbhmGRpc-A2DDn3E~hamnzcOL893tkmRmlbirQElg9M40ylxJzIWGggLRMNadjBJPuvKU5mL59sWPzsgSKmS1WlPBjrs4ZCblSZBH2Y~XOPCJfZE2DH03O1DzNqY48PaTC2CO~n5I2h4mj0gTRDPRFMZz4xVYOqnOdAVCYdT3gXlSLVoUfR4WR6EiCNVC247uSb5Qed3C1f8FHbtrYzJ03EMoN6LZ0rW8yHMgsEIRvWAH408iRb-isjOs3hPHJMyiz8nF4ImnmxTrIY3DlKuoPbMjuEdkFlD5eoRT36oxRV8HOBOulsw~rtvidItq09kzxxoTDGA__&Key-Pair-Id=K34606QXLEIRF3
Example HLS OPUS URL
https://cf-hls-opus-media.sndcdn.com/playlist/d9110040-fe52-4adb-84f0-7d672d7ab077.64.opus/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1vcHVzLW1lZGlhLnNuZGNkbi5jb20vcGxheWxpc3QvZDkxMTAwNDAtZmU1Mi00YWRiLTg0ZjAtN2Q2NzJkN2FiMDc3LjY0Lm9wdXMvcGxheWxpc3QubTN1OCoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA5MTY3MTB9fX1dfQ__&Signature=clc5bSYK96w2VqMHraWqRtjOPEfMMwRJA1U1vCDRbfcwkOdrlMcoxCnW32uShymd0XsSmy0xdcgIASqE1xqjlGfa-aD~~3YEpoFYsiOj3rKPCQf2muowIe4YEj~yUzYm~8ktGA7epSQwLN~oZ5ER6H8vH4vVZP-CNQprNMMZGu0vNQj8TQH2-y0wyKQaN0GgKe1sGOmdyNpWnKAAtqIQvCaC7AoSwSqqMI0HP1rlCUCDtlLEPaXSjVih9SLCjAgNtAA1QTgNhRIfuuJn1AG4iMD4af6mJr5Lskrx90s6OKWcVoJY8Q9KpZo2lCwCK4AR4C7pynz5CqlIThTCBihaCw__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
If you look closely you can see they all share three query parameters:
- Policy
- Signature
- Key-Pair-Id
Policy
This is a base64-encoded JSON object which contains information about the access rules for the resource. Here is one decoded:
{
"Statement": [
{
"Resource": "*/cf-hls-media.sndcdn.com/playlist/jPI4kiRuqQ8N.128.mp3/playlist.m3u8*",
"Condition": {
"DateLessThan": {
"AWS:EpochTime": 1750894776
}
}
}
]
}
DateLessThan is the expiry date as a Unix epoch timestamp. This is always 5 minutes ahead of when the url is called.
In the case of AAC HLS streams, there is also an expires query parameter in the URL, and it isidentical to the value in the Policy JSON.
Signature
This is a cryptographic string used to validate the authenticity of the request. It’s essentially a HMAC or RSA signature applied to the Policy and/or the request URI using a private key. When the SoundCloud CDN receives the request, it uses the corresponding public key to verify the signature hasn’t been tampered with and that the request is within the allowed policy bounds.
If the Signature is incorrect, expired, or doesn’t match the policy, the CDN will reject the request with a 403 Forbidden.
Key-Pair-Id
This is an identifier for the key pair used to sign the Policy. It tells the CDN which public key to use when verifying the signature. In SoundCloud’s case, the ID corresponds to one of their internal key sets.
The problem
Let apiStreamUrl be the URL in transcoding.url, and let contentUrl be the actual CDN stream URL returned from calling apiStreamUrl.
Every time you call apiStreamUrl, it returns the same base contentUrl, but with a different Policy, Signature, and (for AAC) a different expires parameter.
For HLS streams, contentUrl is the m3u8 playlist.
Currently our code only calls apiStreamUrl once to get the m3u8 playlist for HLS streams, and that stays the same for the lifetime of the AudioStream object. The way m3u8 playlists work is that you fetch the m3u8 playlist from contentUrl which has the urls for each chunk of the playlist.
Here is an example:
Example M3U8 playlist
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:1.985272,
https://cf-hls-media.sndcdn.com/media/159660/0/31762/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:2.977908,
https://cf-hls-media.sndcdn.com/media/159660/31763/79410/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:4.989302,
https://cf-hls-media.sndcdn.com/media/159660/79411/159240/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/159241/318900/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/318901/478561/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/478562/638221/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/638222/797882/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/797883/957542/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/957543/1117202/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1117203/1276863/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1276864/1436523/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1436524/1596184/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1596185/1755844/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1755845/1915505/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/1915506/2075165/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:9.978604,
https://cf-hls-media.sndcdn.com/media/159660/2075166/2234825/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXTINF:0.235098,
https://cf-hls-media.sndcdn.com/media/159660/2234826/2238587/jPI4kiRuqQ8N.128.mp3?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1tZWRpYS5zbmRjZG4uY29tL21lZGlhLzE1OTY2MC8qLyovalBJNGtpUnVxUThOLjEyOC5tcDMiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3NTA3ODI5OTh9fX1dfQ__&Signature=Y8mTNc0I4HJkSdxM7aw88PV-p~zAwTA7GrATlX7xiRcd8SsVX7HxLoJLGXQNHopavqiboDrM7wnDIMexriCZHHLfaS5C6xGnMSNFenDFw4GHoXZFahA6Sds05K-Isek3TBpZqOebrtwHJogbiF9fI1E88GrDnGtrdZhpcDct3Abjrhe9kWr~nwb9NEZjQxaUaRhnvRMvq-bs9NsLFNByv5Zk7tYIxgFHLDWfYuZAaTConvFelixiKfK4K5KUhZ7uE2wOPGvNJe1r6T02XgyhF9M7B88ERbxai-fkAodWMLqZhpev64davXfbT7axlB4ChvEXw3WSNtAIuUZlq2PUkQ__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ
#EXT-X-ENDLIST
We pass contentUrl to ExoPlayer, and it fetches this playlist, parses it, and sequentially gets each chunk for playback (it will usually buffer like 10 chunks ahead of the current position).
When NewPipe extracts a SoundCloud stream, it stores it in a StreamInfo. it will also extract the track before and after it for seamless playback once the current song ends. Every time something is extracted it gets put into a cache, and SoundCloud streams have a cache expiry of 5 minutes.
The problem is we only get this playlist once, so the chunk urls are fixed and expire after 5 minutes. This means if you have tracks A, B and C, and you play track B, depending on the length of the tracks, or even if you just pause for more than 5 minutes, ExoPlayer will try to fetch a chunk after it's expired, get a 403 Forbidden, and playback stops and skips to the next track.
Our error handling doesn't handle errors well so it does not retry the track and just skips to the next song. (Reference Player.Java onPlayerError here)
ExoPlayer has no built-in support for refreshing expired HLS playlists. Calling apiStreamUrl returns a new contentUrl (i.e., a playlist URL) every time, and even calling contentUrl will return a fresh m3u8 playlist with updated URLs for each playlist segment; but contentUrl itself expires after 5 minutes.
There's no way to tell ExoPlayer to call apiStreamUrl and then call contentUrl, parse the playlist and update it's internal HlsMediaPlaylist after it is initially created. Even though we pass contentUrl to ExoPlayer for it to parse initially, it only calls it once and never calls it again, so the playlist never gets updated and expires after 5mins because the Policy and Signature are no longer valid.
The Solution
I initially tried to find a solution within ExoPlayer that wouldn’t require changing much NewPipe code, but ExoPlayer exposes no API refresh playlists once they're created. The only way to do it would be to modify the ExoPlayer source code and use our own custom package, but that's too much (and would still be too complicated as well)
So I used a custom HttpDataSource: RefreshableHlsHttpDataSource.
How it works
- When a request to a playlist chunk fails with
403, it means thecontentUrlhas expired, and all the chunk URLs in the playlist have also expired. - At that point, we re-call
apiStreamUrlto get a brand newcontentUrl, which points to a fresh playlist with valid chunk URLs and updatedPolicy,Signature, etc. - We parse this new playlist and construct a mapping between the base paths of the expired chunks and their new equivalents from the fresh playlist. All chunks have the same base path; only the query parameters differ.
- From then on, whenever ExoPlayer tries to load a chunk using an expired URL, it gets mapped to an updated URL. This process repeats when the updated URLs expire.
Other solutions
I spent a lot of time looking for the simplest solution and investigated several approaches.
The ideal solution I was looking for was to replicate browser behaviour: when retrieving a chunk for a track returns 403 Forbidden, reload the entire HLS playlist, get the same chunk, and continue playing the track as normal.
As stated already, ExoPlayer has no mechanism for this out of the box, and it doesn't expose any way to refresh it's internal HlsMediaPlaylist once it gets created. Since this is a problem related to ExoPlayer's lack of functionality to fix this problem, and not necessary related to Player code, I wanted to find a solution that requires minimal changes to Player code and architecture (because the solution wouldn't need to do that if ExoPlayer had an API for refreshing expired HLS playlists).
The main 3 approaches I investigated were:
- Find a way to implement it within ExoPlayer code, such as making a custom version of
DefaultHttpDataSource,DefaultHlsPlaylistTracker,HlsMediaSourceetc., kinda like we have forYoutubeHttpDataSourcewhich is a custom version ofDefaultHttpDataSource. - A level above that: since the 403 is thrown within A
HttpDataSource, then implement solution at that level using a customHttpDataSource - A level above that: solving the problem at the Player level by loading a new media source to ExoPlayer with a fresh HLS playlist and resume from the position of the expired chunk.
Why Solution 1 is infeasible
The internal ExoPlayer code has the initialization and parsing of the HLS playlist spread out through several classes. There's no simple central place I could inject the code to be able to easily refresh the playlist and continue playback as normal. Probably the most practical way to do it would be with reflection, but that is too hacky of a solution and I wanted something cleaner, simple and stable.
Regardless, the only way to know when the playlist expires is from within HttpDataSource#open, so there would still need to be some wiring from within a HttpDataSource via a callback or similar to trigger some code somewhere else that would refresh the playlist, which would introduce coupling with Player code that I'd much rather avoid.
Why Solution 3 is infeasible
Since we can only know when playlist has expired from within HttpDataSource#open, any code that wants to react to that happening needs to be triggered from within that method.
This is because ExoPlayer buffers chunks ahead of time, so it requests expired chunks while it is playing the chunks it has already buffered. It will continue getting 403s for expired chunks up until the playback position reaches the timestamp of that expired chunk, and then it will throw an error and call Player#onPlayerError.
If we react to onPlayerError instead of HttpDataSource#open reloading the playlist, then playback would be stopped for the entire time needed to fetch, parse and load the new playlist, on top of the pause caused by throwing the error. So it makes sense to do it beforehand at the first instance we get a 403 (which is what the browser does).
Therefore, a solution that would load a new media source into ExoPlayer would need to be triggered from within HttpDataSource#open.
A top level of solution at the NewPipe Player level would look like this:
opengets a 403, which triggers a callback- This callback would have to make ExoPlayer play all of it's already buffered content, and then play a new media source starting at the position of buffered content.
We would want to ensure all buffered content gets played because: 1. We don't want to waste that data, and 2. We want to fetch the new HLS playlist while content is playing; otherwise there will be a gap in playback (which the browser doesn't have).
However, ExoPlayer has no way to "hot swap" a MediaSource, and especially not with gapless playback either. Loading a new media source also discards the buffer. The only way to do this would be via polling to check player.getCurrentPosition() and compare with player.getBufferedPosition(), but somehow stop playback before it reaches the end of the buffer (because it will throw an error otherwise) and then load in the new media source; but then that would prevent seamless playback because we wouldn't play til the end of the buffer, and also we wouldn't be loading while the media playing.
The more you delve into it the more complicated it gets, and it would also require changing Player code, because the code currently maps the index of streams in MediaSourceManager.playQueue to their index of the respective MediaSource in the internal ConcatenatingMediaSource playlist in MediaSourceManager.playlist. So adding a new media source to the playlist would invalidate a lot of logic in the code (e.g. 3 != 4 because we'd want playQueue index 3 to == 3, 4, and even == 5 in playlist, if that makes sense) and so a lot of code would need to be changed, and the only way to do it in a clean way would be change the architecture and refactor a bunch of stuff which would be convoluted and non-trivial.
So for these reasons, I abandoned this idea.
Why Solution 2 makes the most sense
This problem is inherently a network issue and so it should be solved at the network level where the error is occurring. If ExoPlayer handled refreshing expired playlists, there wouldn't be any need to change any of our own code: we would just use whatever API was available (like providing a callback to get a new playlist URL or something).
Given this, it made the most sense to implement a solution that maps closest to this ideal scenario that requires minimal changes to Player code.
Due to how ExoPlayer is structured, the most appropriate place to do this is within HttpDataSource#open, as that is what requests the chunks.
Although there's no way to replace ExoPlayer's internal HLS playlist, we can fetch a new playlist from within the HttpDataSource when the internal one expires, and from then on open chunks from the new playlist. This replicates browser behaviour, and has the benefit that the only code we need to add/change is code that is concerned with data extraction.
@litetex I left those TODOs intentionally because I want answers to those questions, and I left it there in the code instead of asking them elsewhere because the person that will see the TODO will have the mind context of the code as they are doing code review, instead of having to context switch into it and then answer the question.
If you know the answer, please feel free to answer it and then I will remove the TODOs
I do not agree that the logging should be removed: I added logging statements in a lot of places to help debug and fix this issue in the first place, and logging doesn't really add/remove functionality or increase reviewing complexity, so I don't see much benefit in removing it and having to manually copy paste those lines into another PR.
On top of that, RefreshableHlsHttpDataSource inherits from LoggingHttpDataSource so I can't exactly remove that from this PR and have the same functionality without doing non-trivial restructuring, and it is not worth the effort to do that just to put it back to normal in a followup PR.