NewPipeExtractor icon indicating copy to clipboard operation
NewPipeExtractor copied to clipboard

Add support for importing Soundcloud likes

Open watermelon42 opened this issue 6 months ago • 1 comments

  • [x] I carefully read the contribution guidelines and agree to them.
  • [x] I have tested the API against NewPipe.
  • [x] I agree to create a pull request for NewPipe as soon as possible to make it compatible with the changed API.

This is to implement https://github.com/TeamNewPipe/NewPipe/issues/3783

The matching NewPipe PR is https://github.com/TeamNewPipe/NewPipe/pull/12367

watermelon42 avatar Jun 18 '25 13:06 watermelon42

It seems the failing tests are unrelated to the changes in the PR

watermelon42 avatar Jun 18 '25 15:06 watermelon42

Can anyone review this pls?

watermelon42 avatar Jul 04 '25 17:07 watermelon42

Thank you for the PR. This works for the example channel posted in the app PR. However, I get following exception when opening the likes tab for https://soundcloud.com/polysormi or https://soundcloud.com/ammaradj:

Stacktrace
java.lang.NullPointerException: text
	at java.util.Objects.requireNonNull(Objects.java:245)
	at j$.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1898)
	at j$.time.OffsetDateTime.parse(OffsetDateTime.java:402)
	at j$.time.OffsetDateTime.parse(OffsetDateTime.java:387)
	at org.schabi.newpipe.extractor.services.soundcloud.SoundcloudParsingHelper.parseDateFrom(SoundcloudParsingHelper.java:139)
	at org.schabi.newpipe.extractor.services.soundcloud.extractors.SoundcloudStreamInfoItemExtractor.getUploadDate(SoundcloudStreamInfoItemExtractor.java:71)
	at org.schabi.newpipe.extractor.stream.StreamInfoItemsCollector.extract(StreamInfoItemsCollector.java:67)
	at org.schabi.newpipe.extractor.MultiInfoItemsCollector.extract(MultiInfoItemsCollector.java:82)
	at org.schabi.newpipe.extractor.MultiInfoItemsCollector.extract(MultiInfoItemsCollector.java:48)
	at org.schabi.newpipe.extractor.InfoItemsCollector.commit(InfoItemsCollector.java:104)
	at org.schabi.newpipe.extractor.services.soundcloud.SoundcloudParsingHelper.lambda$getInfoItemsFromApi$0(SoundcloudParsingHelper.java:401)
	at org.schabi.newpipe.extractor.services.soundcloud.SoundcloudParsingHelper$$ExternalSyntheticLambda4.accept(D8$$SyntheticClass:0)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:203)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:185)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1390)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:475)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:133)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:236)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:435)
	at org.schabi.newpipe.extractor.services.soundcloud.SoundcloudParsingHelper.getInfoItemsFromApi(SoundcloudParsingHelper.java:388)
	at org.schabi.newpipe.extractor.services.soundcloud.extractors.SoundcloudChannelTabExtractor.getPage(SoundcloudChannelTabExtractor.java:73)
	at org.schabi.newpipe.extractor.services.soundcloud.extractors.SoundcloudChannelTabExtractor.getInitialPage(SoundcloudChannelTabExtractor.java:61)
	at org.schabi.newpipe.extractor.utils.ExtractorHelper.getItemsPageOrLogError(ExtractorHelper.java:21)
	at org.schabi.newpipe.extractor.channel.tabs.ChannelTabInfo.getInfo(ChannelTabInfo.java:57)
	at org.schabi.newpipe.extractor.channel.tabs.ChannelTabInfo.getInfo(ChannelTabInfo.java:36)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$getChannelTab$5(ExtractorHelper.java:136)
	at org.schabi.newpipe.util.ExtractorHelper$$ExternalSyntheticLambda0.call(D8$$SyntheticClass:0)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
	at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5377)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)

Not sure if this is related. Is the likes data different from the normal stream info item lists?

TobiGr avatar Jul 07 '25 21:07 TobiGr

I've looked into it and it is caused for users that have liked playlists, not only tracks. The playlist entries don't have a created_at key, hence the NPE.

I'm not sure how to handle this case, but I can think of two options:

  • The easy one, just skip likes of playlists.
  • The hard one, also handle playlists; it's easy to just list them in the Likes tab, but I'd have to add them to the queue recursively when the user taps Play all, it's not done "OOTB".

I'm looking for a way to add them recursively, but if you have other ideas let me know.

Thank you for trying out the code and finding this bug!

watermelon42 avatar Jul 08 '25 13:07 watermelon42

The second one is a limitation of NewPipe as a whole, so please do list playlists in the Likes tab, but don't worry about making them playable with the "play all" button. Thanks :-)

If you want to work on allowing playing lists with mixed playlists and videos, you can do so as a separate PR in the NewPipe app on the refactor branch.

Stypox avatar Jul 08 '25 14:07 Stypox