TeamNewPipe
[SoundCloud] HTTP 403 errors while playing tracks (stream URLs expiration)
Checklist
- [x] I am able to reproduce the bug with the latest version given here: CLICK THIS LINK.
- [x] I made sure that there are no existing issues - open or closed - which I could contribute my information to.
- [x] I have read the FAQ and my problem isn't listed.
- [x] I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
- [x] This issue contains only one bug.
- [x] I have read and understood the contribution guidelines.
Affected version
0.27.6
Steps to reproduce the bug
- Play a soundcloud playlist
- Wait
Expected behavior
All songs play from start to finish
Actual behavior
It will frequently crash with Something went wrong error, and it will skip to the next song, or go back to the start of the song. The crashes seem to follow a pattern as often it will crash at the exact same part of a track every time depending on which track I start playing first. However, since this problem occurs only on 0.27.6 but not 0.27.5 I know this is a NewPipe issue, not a SoundCloud issue.
Screenshots/Screen recordings
No response
Logs
Exception
- User Action: play stream
- Request: Player error[type=ERROR_CODE_IO_BAD_HTTP_STATUS] occurred while playing https://soundcloud.com/lusperae/avid-modv-sawanohiroyuki-nzk
- Content Country: GB
- Content Language: en-GB
- App Language: en_GB
- Service: SoundCloud
- Timestamp: 2025-02-24T20:29:33.506Z
- Package: org.schabi.newpipe
- Service: SoundCloud
- Version: 0.27.6
- OS: Linux Android 9 - 28
Crash log
com.google.android.exoplayer2.ExoPlaybackException: Source error
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:644)
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:616)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:214)
at android.os.HandlerThread.run(HandlerThread.java:65)
Caused by: com.google.android.exoplayer2.upstream.HttpDataSource$InvalidResponseCodeException: Response code: 403
at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.open(DefaultHttpDataSource.java:413)
at com.google.android.exoplayer2.upstream.DefaultDataSource.open(DefaultDataSource.java:263)
at com.google.android.exoplayer2.upstream.TeeDataSource.open(TeeDataSource.java:52)
at com.google.android.exoplayer2.upstream.cache.CacheDataSource.openNextSource(CacheDataSource.java:796)
at com.google.android.exoplayer2.upstream.cache.CacheDataSource.open(CacheDataSource.java:609)
at com.google.android.exoplayer2.upstream.StatsDataSource.open(StatsDataSource.java:84)
at com.google.android.exoplayer2.source.hls.HlsMediaChunk.prepareExtraction(HlsMediaChunk.java:495)
at com.google.android.exoplayer2.source.hls.HlsMediaChunk.feedDataToExtractor(HlsMediaChunk.java:468)
at com.google.android.exoplayer2.source.hls.HlsMediaChunk.loadMedia(HlsMediaChunk.java:437)
at com.google.android.exoplayer2.source.hls.HlsMediaChunk.load(HlsMediaChunk.java:394)
at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:412)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)
Affected Android/Custom ROM version
Android 9
Affected device model
Samsung Galaxy S8
Additional information
This is NOT a duplicate of #9925. Although the repro steps are identical, that one happens occasionally, does not exhibit the same behaviour or patterns, and has been present for several years, so most likely does not have the same root cause or fix.
Whereas this problem only appeared on 0.27.6 and happens constantly, predictably, and annoyingly. Using 0.27.5 does not have the problem, but then of course I can't watch YT videos on 0.27.5 because it doesn't work :)
I am having this same issue with the latest stable release, as well as on the latest stable release of Tubular. I think this error also occurs with enqueued Youtube videos, and also with auto-enqueue playlists. If you say that an earlier version doesn't have this issue, a temporary solution (or permanent, if you like the idea) might be to keep Soundcloud subscriptions and playlists on the older Newpipe and continue watching Youtube on the latest fork, like Tubular.
Putting this here just so it's visible: this bug was introduced by 1269 in the extractor repo when track_authorization and HLS streams were added.
So after a lot of debugging and analysing Soundcloud in devtools, I have found the cause of https://github.com/TeamNewPipe/NewPipe/issues/12109 which was introduced by https://github.com/TeamNewPipe/NewPipeExtractor/pull/1269/
Currently, when we extractor info for a soundcloud url, the extracted JSON contains part of an API url for each transcoding in media.transcodings[].url
Example:
"transcodings": [
{
"url": "https://api-v2.soundcloud.com/media/soundcloud:tracks:843895900/7b7c52ff-ddca-42bd-8692-21fd14a34e85/stream/hls",
Using the client_id and track_authorization, we call this endpoint to get the stream url for that transcoding.
Example request:
https://api-v2.soundcloud.com/media/soundcloud:tracks:690040873/67cae8d9-7363-4aff-8003-2e905b315f86/stream/hls?client_id=vjvE4M9RytEg9W09NH1ge2VyrZPUSKo5&track_authorization=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJnZW8iOiJHQiIsInN1YiI6IiIsInJpZCI6IjhlNzIxMDhmLWRmNzctNDAwZC05OGM1LTQ0YzRhYjliZDJjOSIsImlhdCI6MTc0OTYzMjE1Mn0.d0DzxofzLlnLqiwSPEXwvuzundSHDbeLJQmYAIMbQ1U
Example response:
{
"url": "https://playback.media-streaming.soundcloud.cloud/jPI4kiRuqQ8N/aac_160k/67cae8d9-7363-4aff-8003-2e905b315f86/playlist.m3u8?expires=1749632412&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbGF5YmFjay5tZWRpYS1zdHJlYW1pbmcuc291bmRjbG91ZC5jbG91ZC9qUEk0a2lSdXFROE4vYWFjXzE2MGsvNjdjYWU4ZDktNzM2My00YWZmLTgwMDMtMmU5MDViMzE1Zjg2L3BsYXlsaXN0Lm0zdTg~ZXhwaXJlcz0xNzQ5NjMyNDEyIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzQ5NjMyNDEyfX19XX0_&Signature=ZKZRPdm3PEGlF0fyGtjqG2hv6max4UO2GzzWHb3u3THOVN8ehfIcjQl2a0Z-Fwg-rbVH1hQeceev~o6XevdpXMOr488kbVdUAOdTTVF0hYoL4Rrwp5gX6OqzNpltSQHZXVYvFo2R1iBLab2kyOi2rgLEKNM-duoiyUTG7I0ooO162QD7P6nH9Aa85tVUA16l7RminPqZCHfeMHHGSvTcA2U~KyNFmQOyy5Z4VJHrrtMOSNQXvxa6UU8KbW~xk2gXdQswuGSiVnL3q9XA4YZyzu0DvnzB8y2Ef2gDF~XDnLIjzuPso95VkAcVbmiaf6X-IA7bf0HV1gipePZSpfPtOw__&Key-Pair-Id=K34606QXLEIRF3"
}
I know the problem is caused by a combination of using track_authorization and now having a HLS mp3 stream. Idk if passing track_authorization now means the stream urls expire and the expiry is failing an integrity check server side and with the track_authorization, or it's simply an issue to do with hls mp3 stream.
In any case, each stream url has an expiry date.
These seem to be about 5 minutes in the future. From what I see in devtools, the browser uses the stream url to get the segments of the m3u8 playlist.
The stream urls in the form of playback.media-streaming.soundcloud.cloud/5vLcDIUM82Pr/aac_160k are for aac format, and always have an expires parameter.
However, we only extract audio streams for mp3 and opus. (Anyone know why this is?)
These have the form https://cf-hls-media.sndcdn.com/playlist/jPI4kiRuqQ8N.128.mp3/playlist.m3u8? and do not have an expires parameter, but they still expire nonetheless.
From what I can see in devtools, the client uses the stream url to periodically fetch chunks of the playlist and plays it. Eventually fetching chunks from that url returns 403, and it uses the api endpoint to get another stream url and fetches using that.
I cannot see anywhere in our code where we do this. We extract audio streams once for a url to create AudioStreams for a StreamInfo (hls mp3, progressive mp3, and opus), and the transcoding urls stored for each AudioStream are static for the lifetime every StreamInfo object.
Since the player fetches streams for previous and next track to achieve seamless playback, it will have a StreamInfo for the next track in memory for the entire duration of the current track. However, for soundcloud streams these only stay in cache for 5 minutes.
Track 1 05:17 Track 2 04:04 Track 3 03:56
I have tracks in my playlist with these durations, and I can consistently replicate a 403 whenever I play track 1 on fresh app start with empty cache. When I get to 0249 in track 3 I get a 403 every single time.
This is dependent on how many chunks of the playlist ExoPlayer fetches before the m3u8 playlist urls expire. If it fetches all the chunks before the end of the track then it can play tracks longer than 5 minutes. It also depends on the length of the tracks with respect to how long SoundCloud StreamInfo's stay in the cache (which is 5mins). I don't know the internals of ExoPlayer and how it handles prefetching for HLS m3u8 playlist streams.
In DevTools it seems it's a fetching a lot more of the track before it plays it than we do in NewPipe. It gets the first 30 chunks, which equates to 300 seconds of playback as each segment is 10s. 2:49 would be 25 chunks.
Anyway, regardless, the correct solution would be to refactor how we use the ExoPlayer to implement retry logic to refetch transcoding urls for SoundCloud streams where we HTTP 403.
@Stypox what's the word on this? Might try to take a stab at it as it's affecting me often
@mjsir911 I have already fixed it in a PR which is linked to this issue