Add Invidious support

[litetex]

What is it?

• [x] Feature (user facing)

Description of the changes in your PR

👉 https://github.com/TeamNewPipe/NewPipeExtractor/pull/833

Adds support for Invidious instances (require a working API).

• Divided YouTube into "Youtube-Like" services, currently YouTube itself or Invidious and allows seamless switching between them:
• Created a new "services" section inside the "content"-settings
• Added/Abstracted a configuration area for instance-based services (used inside YouTube and PeerTube) + Created/Improved the configuration-area for YT-Like (and also PeerTube) instances:
• Cherry picked commits from https://github.com/TeamNewPipe/NewPipe/pull/7458 that fix the placeholders; without these fixes the icons for YT and invidious are e.g. always white
  • Also updated the YT placeholder accordingly, as discussed internally.
• Abstracted a ton of code so that it's easy to maintain and extendable in the future (e.g. it's now easily possible to also add other YT proxies like Piped)

Configuration demo:

https://user-images.githubusercontent.com/40789489/163622042-d18894f0-6b5c-4b7a-93a3-7dcaceac7bc9.mp4

Fixes the following issue(s)

• Fixes https://github.com/TeamNewPipe/NewPipeExtractor/issues/427

Relies on the following changes

• https://github.com/TeamNewPipe/NewPipeExtractor/pull/833

APK testing

The APK can be found by going to the "Checks" tab below the title. On the left pane, click on "CI", scroll down to "artifacts" and click "app" to download the zip file which contains the debug APK of this PR.

Due diligence

• [x] I read the contribution guidelines.

Todo

• [x] Add a help page explaining what e.g. invidious / YouTube-like is.
• [x] Check if throttling occurs - Couldn't find/Notice any problems so far
• [x] Dislikes are not correct (always displays 0 dislikes)
• [ ] Date of related items is missing ⚠️ Not provided by API → https://github.com/iv-org/invidious/issues/3058
• [x] Videos have the thumbnail of the channel (Player-Notfication)
• [x] Modify YT icon
• [x] Add a note that Invidious instances require a working API
• [ ] Test (NewPipeExtractor)
• [ ] Links from non known instances are not redirected (new issue?)
• [ ] Duplicate feed/history items when using switching between instances → requires a major rework of the internal data-structure (currently only based on urls)

[TheFrenchGhosty]

Please link to https://instances.invidious.io/ instead, since this will always link to the instances list.

PS: good job

[litetex]

The icon you are now using for the YouTube service is close to the one which is used by YouTube on their platform. By seeing that Vanced has been takedown because of their logo very close to the one of YouTube (this is what the Vanced team said), we should not take the risk at all to use a similar logo to the YouTube one, otherwise this will open a door for Google to DMCA the app.

@TiA4f8R This is the NewPipe logo inside a rectangular box with rounded corners. It' doesn't even use the same color and the triangle is not even filled and closed. When you start the app and see the loading screen you see the same logo inside a rectangular box with rounded corners (your phone) with a red background (IMHO this resembles the YT logo closer than the above). If that is license infringement then maybe YT should consider removing the unicode symbol U+25B6 aka ▶ which was created in 1993 - 12 years before YT was founded.

I'm also very skeptical about Vanced's explanation: Shutdown because of the logo? Why didn't they simply change it? Looks to me like a scapegoat reason...

We can maybe change it to a completely rectangular box without rounded corners if that's ok for you, but I definitely won't remove the triangle.

[Myzel394]

@litetex This looks very promising! After that long time, someone finally got the courage to implement this feature! :)

Here are my two cents about what I think should be changed a little bit: I think it would be nice to have a button to automatically load the official instances listed on https://api.invidious.io/ (we should probably sort them by -health) And maybe a brief explanation about "Youtube-Like instances" would be helpful for people who don't know anything about Indivious.

Anyways, thanks for your contribution!

[TheFrenchGhosty]

@Myzel394

I think it would be nice to have a button to automatically load the official instances listed on https://api.invidious.io/ (we should probably sort them by -health)

We don't recommend using anything other than instances.invidious.io since there's plan to rework api/redirect...

[Myzel394]

@Myzel394

I think it would be nice to have a button to automatically load the official instances listed on https://api.invidious.io/ (we should probably sort them by -health)

We don't recommend using anything other than instances.invidious.io since there's plan to rework api/redirect...

Okay thanks for the hint. I'm a little bit confused since https://api.invidious.io/ is literally linked on https://instances.invidious.io. However, are there other api endpoints to fetch a list of available instances?

[TheFrenchGhosty]

@Myzel394

Okay thanks for the hint. I'm a little bit confused since https://api.invidious.io/ is literally linked on https://instances.invidious.io/. However, are there other api endpoints to fetch a list of available instances?

Sorry, I badly explained (and forgot to reply sooner): what I mean is that we don't recommend using anything other than instances.invidious.io as a web interface that users interact with, the json endpoint will always remain (it's made to be stable), but we have plan to make a single web interface (on instances.invidious.io ) to replace both the current markdown file (located in the documentation) and the web interface of the instance api (and ideally redirect.invidious.io), this is why we recommend using instances.invidious.io (and not link to the markdown file directly, or another web interface).

[opusforlife2]

I can't seem to add invidious.kavin.rocks, though invidious.snopyta.org works.

[unixfox]

I can't seem to add invidious.kavin.rocks, though invidious.snopyta.org works.

Usually only the invidious instances with API available will work: https://api.invidious.io/

[opusforlife2]

@FireMasterK Is it possible to add the API?

[FireMasterK]

@FireMasterK Is it possible to add the API?

No, they've been blocked due to bots abusing them.

If there's a way to whitelist NewPipe, I may consider it however.

Currently for the time being, only piped allows all traffic.

[opusforlife2]

@litetex Just like you have made it so that the service and instance are shown clearly for Invidious (Invidious / instance-here), could you also add that bit for Peertube? (PeerTube / instance-here)

It would easily fix #8013.

[opusforlife2]

Check if throttling occurs

I'm seeing a lot of this. Does the APK use Googlevideo servers finally, or is it proxied through the Invidious server?

[litetex]

@opusforlife2

@litetex Just like you have made it so that the service and instance are shown clearly for Invidious (Invidious / instance-here), could you also add that bit for Peertube? (PeerTube / instance-here)

It would easily fix https://github.com/TeamNewPipe/NewPipe/issues/8013.

This should be done inside a new PR - if it's actually required/wanted. I added it for Invidious because it's easier to see what kind of YT-like service you are using.

I'm seeing a lot of this. Does the APK use Googlevideo servers finally, or is it proxied through the Invidious server?

As far as I know the steams are fetched directly from YT. I have to rebase the PR to get the latest changes, which might fix the problems.

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

0.0% Coverage
1.2% Duplication

[opusforlife2]

BTW, what about fetching the actual list of official instances from the Invidious URL and letting the user tap them to add them to Newpipe? Much more user-friendly than manually entering a URL. This way, the user will also avoid mistakenly adding an instance that doesn't provide the API as needed.

Edit: Should the Youtube entry be removable? I feel like that might lead to accidental removals by novice users, who won't understand how to get it back without creating undue noise in our repo. The button to restore defaults is right there, but you know how most users are.

Edit 2: Is everything fetched through Invidious? As in, do we avoid pinging Youtube at all? It seems like that, because I just refreshed my feed and it was much slower. (Feed is definitely loaded from Invidious, but I don't know about pinging YT at all.)

~~Edit 3: OHMYGAWD the Invidious APK doesn't load all the livestreams to the feed that the official one does. It might be a bug, but for now, no more livestreams from hell! I'm always gonna update the feed after switching to Invidious, even if I use YT later.~~ Seems to have been temporary.

Edit 4: I forgot to mention: the thumbnail when using Invidious is smaller. It doesn't reach the device edges. There are white/black (based on theme) bars on either side of it. Switching to YT loads the same thumbnail properly.

Edit 5: The Description tab text is not formatted when the video details are loaded from Invidious. It's a giant continuous paragraph. Also, live videos aren't marked as such in an Invidious-loaded feed.

[ghost]

not quite sure where else to put this so ill ask here: maybe this is a config issue on my end, but it seems theres an issue with automated subtitles? does invidious not support those? if not, okay, but if they do im curious as to why it doesnt show up in your NP fork

edit: yea they totally support automated subtitles, at least invidious.tiekoetter.com, so why doesnt it work in NP? hm

[adamboutcher]

I can't find the test apks in the checks, have the artifacts expired?

[bigdiff]

I can't find the test apks in the checks, have the artifacts expired?

I can still download the latest artifact build

[adamboutcher]

I can't find the test apks in the checks, have the artifacts expired?

I can still download the latest artifact build

Thanks, turns out the GH app doesn't show everything...

[qdrop17]

Hi there.

I'm eager to test the apks, but they are no longer available:

What's still open to merge this PR? It's an awesome feature indeed 😄.

[Myzel394]

Are there any news on this pr? I'm also waiting for the update 😁

[Stypox]

Although this was a really good PR, I'm closing it as the author is not going to contribute to NewPipe anymore. Thanks to everyone involved, we'll try to implement this feature in the future rewritten NewPipe!

[Myzel394]

Although this was a really good PR, I'm closing it as the author is not going to contribute to NewPipe anymore. Thanks to everyone involved, we'll try to implement this feature in the future rewritten NewPipe!

Sorry for off topic but where can we find that project - or is rewriting Newpipe just planned yet?

[opusforlife2]

Just planned.

[Quantum-Tinkerman]

Been planned for years now 🤣