NewPipeExtractor
NewPipeExtractor copied to clipboard
TeamNewPipe
Attempt to parallelize YouTube Stream extractor
- [x] I carefully read the contribution guidelines and agree to them.
- [ ] I have tested the API against NewPipe.
- [ ] I agree to create a pull request for NewPipe as soon as possible to make it compatible with the changed API.
- Are there currently some performance problems that would warrant such a change and its developer overhead?
- If there are, how does it look like with this change. Any statistics or the like?
ps: I did NOT take a real look at the code
Why use Java futures manually? I'd rely on RxJava instead (like we already do in the app)
Why use Java futures manually? I'd rely on RxJava instead (like we already do in the app)
I think there is no RXJava inside this project.
I think there is no RXJava inside this project.
Yeah, I know. I'd propose to use it, what do you think?
Yeah, I know. I'd propose to use it, what do you think?
I don't know.
- RxJava was built for event based programming however there are no events here. Just asynch tasks.
- It would add a new dependency here; (Larger) dependencies tend to introduce problems...
- I might be biased because personally the most RxJava code I read so far in NewPipe was horribly complex or caused problems that have been different for debugging
I'm also fine with the current way of using the java native way. However before we get technical 👉 https://github.com/TeamNewPipe/NewPipeExtractor/pull/751#issuecomment-971826701
Was there any conclusion to what library to use? Should we go with the default Java API? I'd like to rework on this, one last time.
Are there currently some performance problems that would warrant such a change and its developer overhead?
Yes, currently in NewPipe's case, a minimum of 2 requests are made regardless. We parallelize them, to reduce the time taken by half. Another additional request may be made - ios or android client, depending if it is a live stream or not.
In Piped's case, a minimum of 4 requests are made since I force the android and ios player fetch.
If there are, how does it look like with this change. Any statistics or the like?
Piped's response times have dropped by ~400-500ms on average (it depends on how fast youtube's server responds), this should benefit NewPipe too.
I tried updating the mocks, but it appears to be unable to find some requests, anyone have any idea why this might be happening?
It's still a draft since it would have issues with mocks.
What I'm going to do to fix them is remove the cpn variable before checking if a mock request matches a response.