invidious
invidious copied to clipboard
iv-org
[Bug] The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
For everyone:
We are fully aware of this issue, please comment only if you have new info to bring on. Please refrain from opening new issues. YouTube has recently started to care about alternative frontends like Invidious and is trying to block them. This currently only affects some public Invidious instances, so if you have the resources we recommend installing Invidious on your own personal computer or server: https://docs.invidious.io/installation/
For Invidious instance maintainers:
Please read this comment for a temporary solution: https://github.com/iv-org/invidious/issues/3822#issuecomment-1565401621
If you want to discuss or read for other solutions: https://github.com/iv-org/invidious/issues/3915
BUG: The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
When trying to loading any video on the invidious.vpsburti.com instance, this error is shown:
Title: `The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)`
Date: `2023-05-27T08:21:28Z`
Route: `/watch?v=Nx-ougrNm50&listen=1`
Version: `2023.05.14-3a54e95 @ master`
<details>
<summary>Backtrace</summary>
<p>
The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
from /usr/share/crystal/src/array.cr:114:31 in 'extract_video_info:video_id'
from src/invidious/videos.cr:377:10 in 'fetch_video'
from src/invidious/videos.cr:365:13 in 'get_video:region'
from src/invidious/routes/watch.cr:63:15 in 'handle'
from lib/kemal/src/kemal/route.cr:13:9 in '->'
from src/invidious/helpers/handlers.cr:30:37 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/init_handler.cr:12:7 in 'process'
from /usr/share/crystal/src/http/server.cr:500:5 in '->'
from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
from ???
</p>
</details>
It appears that YouTube is doing some breaking changes in a rollout way, which means it doesn't affect all the instances.
I'm trying to find an affected IP address where I could debug the issue. For now it seems like people that are hosted at hetzner (invidious.nerdvpn.de & invidious.vpsburti.com) are affected.
I've got this as well
Title: The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
Date: 2023-05-27T06:08:04Z
Route: /watch?v=QncYBP785uA
Version: 2023.05.25-381a0e3 @ master
Backtrace
The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
from /usr/share/crystal/src/array.cr:114:31 in 'extract_video_info:video_id'
from src/invidious/videos.cr:377:10 in 'fetch_video'
from src/invidious/videos.cr:365:13 in 'get_video:region'
from src/invidious/routes/watch.cr:63:15 in 'handle'
from lib/kemal/src/kemal/route.cr:13:9 in '->'
from src/invidious/helpers/handlers.cr:30:37 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call'
from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/init_handler.cr:12:7 in 'process'
from /usr/share/crystal/src/http/server.cr:500:5 in '->'
from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
from ???
You can solve this issue by changing the IP address of your server.
Nowadays, you get IPv6 on most server providers. Try curl -I -6 ipv6.google.com and if you don't get any error, you have ipv6. If not, try to get a new IPv4 address.
If you are on Docker
Very easily solution based on the docker-compose provided in the documentation
Adapt this example docker-compose to your current docker-compose configuration:
version: "3"
services:
ipv6nat:
container_name: ipv6nat
privileged: true
network_mode: host
restart: unless-stopped
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:ro'
- '/lib/modules:/lib/modules:ro'
image: robbertkl/ipv6nat
invidious:
image: quay.io/invidious/invidious:latest
# image: quay.io/invidious/invidious:latest-arm64 # ARM64/AArch64 devices
restart: unless-stopped
networks:
- invidious
ports:
- "127.0.0.1:3000:3000"
environment:
# Please read the following file for a comprehensive list of all available
# configuration options and their associated syntax:
# https://github.com/iv-org/invidious/blob/master/config/config.example.yml
INVIDIOUS_CONFIG: |
db:
dbname: invidious
user: kemal
password: kemal
host: invidious-db
port: 5432
check_tables: true
# external_port:
# domain:
# https_only: false
# statistics_enabled: false
healthcheck:
test: wget -nv --tries=1 --spider http://127.0.0.1:3000/api/v1/comments/jNQXAC9IVRw || exit 1
interval: 30s
timeout: 5s
retries: 2
logging:
options:
max-size: "1G"
max-file: "4"
depends_on:
- invidious-db
invidious-db:
image: docker.io/library/postgres:14
restart: unless-stopped
networks:
- invidious
volumes:
- postgresdata:/var/lib/postgresql/data
- ./config/sql:/config/sql
- ./docker/init-invidious-db.sh:/docker-entrypoint-initdb.d/init-invidious-db.sh
environment:
POSTGRES_DB: invidious
POSTGRES_USER: kemal
POSTGRES_PASSWORD: kemal
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
volumes:
postgresdata:
networks:
invidious:
name: invidious
enable_ipv6: true
ipam:
config:
- subnet: fd00:dead:beec::/48
The major changes are networks:, adding invidious and invidious-db to the invidious network and adding the service ipv6nat.
Then if execute this command and if you get a ping reply then everything is working fine:
docker compose exec -u root invidious ping -c 2 ipv6.google.com
The downside of this option is that you are running a separate service in privileged mode, and this is not great for security. If possible, I would recommend following the official documentation below.
Or the official way using the Docker documentation for IPv6
Or you can also take a look at the official Docker documentation for IPv6, which explains the official way to configure IPv6 on Docker:
- https://docs.docker.com/config/daemon/ipv6/
- https://gdevillele.github.io/engine/userguide/networking/default_network/ipv6/
Not on Docker
If you already have IPv6, then you can try to use IPv4 by switching this parameter in the config.yml (found here):
force_resolve: ipv4
Or try the tips below about ipv6.
Apply for both Docker and not Docker
If you can replicate the issue again on IPv6, try to add a new IPv6 address. Use ip a and check if your IPv6 address doesn't end with /128 then you are in luck.
You can then try to add a new IPv6 address using this tutorial: https://tldp.org/HOWTO/Linux+IPv6-HOWTO/ch06s02.html. All you want to do is to change just a number or a letter, example:
2001:0db8:0:f101::1/64 to 2001:0db8:0:f101::2/64
Or you can try to switch back to IPv4 by switching this parameter in the config.yml (found here):
force_resolve: ipv4
After a bit more dinging into the issue, I've found that even on the official www.youtube.com website you get the error:
I'm not sure if it's an error on their side or a way to "shadow ban" an IP address. For now I don't have a solution except using another IP address like described above.
Another instance that started having the same problems a couple of hours ago: invidious.nogafa.org Judging by Whois, it is hosted on OVH SAS. This state of affairs is already starting to worry me, because if Google continues to ban VPS/VDS providers, then I'm afraid that over time, every Invidious instance will not work anymore.
Started noticing the same errors today as well at https://invidious.sethforprivacy.com
Not sure if this is related, but https://yewtu.be started giving empty videos with "This content is not available", tested on multiple videos from different channels.
https://y.com.sb and https://inv.vern.cc seem affected from the original error.
Not sure if this is related, but yewtu.be started giving empty videos with "This content is not available", tested on multiple videos from different channels.
y.com.sb and inv.vern.cc seem affected from the original error.
Yes it's related.
I've temporarily fixed the issue on yewtu.be by applying the solution in https://github.com/iv-org/invidious/issues/3822#issuecomment-1565401621, but there is a big probability that the issue will arise again in a few hours.
Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?
Could it be useful to have a script that automatically changes the IPv6 address and hook it up to a cronjob?
There is a script just for that :D: https://github.com/ycd/ipv6-rotator
But I did not test if changing the IPv6 address in the same range fixes the issue or not.
@unixfox over on yt-dlp discord, the one user who's getting the new "this content isn't available" video claims to still be able to access youtube in a browser. As such, next time yewtu.be breaks, can you please check whether it can still access youtube in a browser?
This video is available on YT from my home’s computers, but not on my favorite Invidious instance (FDN’s).
@unixfox over on yt-dlp discord, the one user who's getting the new "this content isn't available" video claims to still be able to access youtube in a browser. As such, next time yewtu.be breaks, can you please check whether it can still access youtube in a browser?
I use Invidious in Edge on my Xbox. It is having the exact same problem. The odd instance will work if you keep going through the list but thats a huge pain in the ass.
@unixfox over on yt-dlp discord, the one user who's getting the new "this content isn't available" video claims to still be able to access youtube in a browser. As such, next time yewtu.be breaks, can you please check whether it can still access youtube in a browser?
@gamer191 like in https://github.com/iv-org/invidious/issues/3822#issuecomment-1565457284I I'm getting the new video saying "content unavailable" on www.youtube.com.
Hmm, after refreshing the page on this bug 2-4 or more times the video n audio responds and playback works fine for whole duration.
Details
Title: `The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)`
Date: 2023-06-09T12:36:14Z
Route: /watch?v=PQ_dmFUR2N8&nojs=1
Version: 2023.06.04-c8b7bf2 @ master
The video returned by YouTube isn't the requested one. (WEB client) (VideoNotAvailableException)
from /usr/lib/crystal/core/json/any.cr:102:10 in 'extract_video_info:video_id'
from /usr/lib/crystal/core/hash.cr:1185:13 in 'fetch_video'
from src/invidious/videos.cr:357:17 in 'get_video:region'
from src/invidious/routes/watch.cr:63:15 in 'handle'
from lib/kemal/src/kemal/route.cr:13:9 in '->'
from lib/kemal/src/kemal/config.cr:92:7 in 'call'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call'
from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next'
from lib/kemal/src/kemal/init_handler.cr:12:7 in 'process'
from /usr/lib/crystal/core/http/server.cr:500:5 in '->'
from /usr/lib/crystal/core/fiber.cr:146:11 in 'run'
from /src/boringssl/src/google-boringssl-251b516/ssl/internal.h:334:8 in '??'
Not sure if this is related, but yewtu.be started giving empty videos with "This content is not available", tested on multiple videos from different channels.
Yes it's related.
I've temporarily fixed the issue on yewtu.be by applying the solution in #3822 (comment), but there is a big probability that the issue will arise again in a few hours.
That seems to be the thing happening with Piped: https://github.com/TeamPiped/Piped/issues/2487
Google is doing something VERY sussy atm
This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. :middle_finger: <---- that's for you Google
If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.
This hit me hard. OVH only gives you a /128 block for the IPv6 space, so I could mitigate this for a while using that, but it would be pointless if I'm going to get IP blocked again on IPv6. 🖕 <---- that's for you Google
You could try getting a proper IPv6 range from tunnel broker such as Hurricane Electric.
I'm getting this issue too and it's quite breaking, most instances just don't work, there are some that do.
Could this be related to the recent Youtube takedown thing?
I'm getting this issue too and it's quite breaking, most instances just don't work, there are some that do.
Could this be related to the recent Youtube takedown thing?
Basically, it appears that Google is implementing an IP ban on Invidious hosts, but it can be bypassed by using a random IPv6 address. I find it somewhat strange that IP bans are still being used nowadays; it's a technique reminiscent of a kid running a Minecraft server.
Suggestion for your sanity: specialcase the Invidious page for this error so it links to this issue rather than to a new issue submission form (at least temporarily).
I've had success routing outbound invidious traffic over shared VPNs; blending traffic with 'legitimate' users and connecting from non-Hetzner/OVH/DO address space works well for now.
PostUp = ip route add default dev %i table 5280
PostUp = ip -6 route add default dev %i table 5280
PostUp = ip rule add from (wg v4 address) table 5280
PostUp = ip -6 rule add from (wg v6 address) table 5280
PostUp = ip -4 rule add uidrange 991-991 lookup 5280
PostUp = ip -6 rule add uidrange 991-991 lookup 5280
PostDown = ip rule del from (wg v4 address) lookup 5280
PostDown = ip -6 rule del from (wg v6 address) lookup 5280
PostDown = ip -4 rule del uidrange 991-991 lookup 5280
PostDown = ip -6 rule del uidrange 991-991 lookup 5280
Table = off
reCAPTCHA profiles addresses at the /64 so it probably won't be long until we see that on YT as well, if they aren't already.
I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?
I'm not smart when it comes to networking, where do you add these lines? Also, how do you authenticate with WG? You can't just connect to the IP without a private key right?
To keep wireguard from updating the system's default routing table we disable that functionality with Table = off, create our own table called 5280, add wireguard's addresses, and create a rule to lookup table 5280 for traffic which originates from the user running invidious, in my case the user ID is 991. These lines are added to the Interface section.
If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.
Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.
If there's some sort of way to set up invidious with a SOCKS5 proxy, I could probably rotate IPs on DigitalOcean/Vultr.
Double that. You could also use a “residential IP” proxy service, or even one of the mobile proxy ones (which use mobile operators networks and usually change IPs every couple minutes for you). This might not work well with proxying video traffic though, and I'm afraid it also might be not too cheap nowadays.
Are you sure the video traffic has to be proxied at all?
I have created a separate issue for freely discussing how to circumvent the current blockage: #3915.
This way people subscribed in this issue will not be "spammed" and we can keep this issue for tracking the new ways to really solve the original issue, not temporary workarounds.
Please refrain from talking about temporary workarounds here and use #3915 instead. Thank you
Closing as youtube is now blocking projects based on it using other methods: https://github.com/iv-org/invidious/issues/4045
Seems like this is back. Happening on multiple instances, across multiple videos. This started happening around 2AM EST today. Eventually if I switch instances enough there are a few which are not blocked.
Instances which are blocked: https://par1.iv.ggtyler.dev/ https://invidious.io.lol/ https://invidious.no-logs.com/ https://yt.cdaut.de https://vid.priv.au https://invidious.einfachzocken.eu/ https://onion.tube/ https://yt.oelrichsgarcia.de
Instances which work: https://invidious.fdn.fr/ https://invidious.lunar.icu https://invidious.nerdvpn.de https://inv.us.projectsegfau.lt
