invidious
invidious copied to clipboard
iv-org
Add support for using Invidious through a HTTP Proxy
Partially addresses #301
To use just set at least the host and port configuration here in the Invidious configs.
##
## Configuration for using a HTTP proxy
##
## If unset, then no HTTP proxy will be used.
##
http_proxy:
user:
password:
host:
port:
Now that there's "proper" support for HTTP proxies, I suppose Invidious should also be removing its own proxy implementation at HTTPClient and HTTPProxy
Won't that collide with the code in src/invidious/yt_backend/proxy.cr?
It might be a good idea to remove all of that proxy code (the IP list hasn't been updated in ages anyway) and let the instance owner use their own.
I think it might be helpful to add the http proxy for pulling in reddit comments as well, provided it s not alot more work on top of this.
Reddit just recently IP banned alot of VPS IP ranges, so currently the easiest work around is to tunnel things either via TOR or cloudflare warp
Is this patch meant to also address the install step that says
Checking player dependencies, this may take more than 20 minutes... If it is stuck, check your internet connection.
?
Edit: It might. At least this time the error is about npm, and I can probably figure out how to configure npm's proxy settings inside. Edit 2: Seems we don't use proper npm but instead download directly in fetch-player-dependencies.cr line 86. Edit 3: With exotic network setups we shoud just use proper npm for downloading: #4361
The part that fetches the videojs dependencies is technically not a part of the main Invidious program so requests in that don't go through the configured proxy.
The part that fetches the videojs dependencies is technically not a part of the main Invidious program so requests in that don't go through the configured proxy.
Maybe we should? If one configure a proxy he expect all the requests to go through the proxy.
I think eb21454849f1cd19b47c6d31b3a2365469468f95 and 9a5a743b334b07f49868726a73d0ddb703bce2e9 should both be amended into their parent commits.
If one configure a proxy he expect all the requests to go through the proxy.
A lot of fetch-player-dependencies.cr is just calling OS commands already, so it may be easier to just invoke wget or curl rather than implementing proxy support in its crystal parts. I even wonder if it might be worth converting the script to bash entirely.
Please use the edit button, don't triple post...
Please don't go offtopic, create a separate issue for your npm discussion.
I researched a bit more about how to best pass proxy settings to the build process and found that doing it securely (not revealing proxy hostname and credentials in the built image) is more complicated than I had hoped. So build-time proxy stuff should best be left for a 3rd-party project.
@syeopite, I found a few more things that I'd prefer switched around in git history, and some whitespace that seems accidential. Please allow me to propose this alternative history for the same effect as your current PR. The last commit is just so you can easily verify that files in both branches are otherwise identical. If you like, feel free to hard-reset your branch to my proposal's 2nd commit.
Does Crystal have an elegant way to use a common function to de-duplicate the conn.proxy = … assignments?
A project-wide function to set default config for all HTTP(S) connections might also turn out to be useful if one day we want to configure more aspects than just proxy.
@syeopite https://github.com/mamantoha/http_proxy/blob/master/CHANGELOG.md#0102 I think you might be able to update the library to a newer version for the crystal 1.12 support
Hello, I just tested this PR on my (was public, now blocked by YT) instance to check whether I can use it to bypass the IP-blockage.
I found out that it works for the most part but there are problems with the amount of connections to the HTTP proxy server (used a simple privoxy) on a instance with many users, especially the pubsub feeds update caused problems and hasn't (yet) fully worked using this PR. I had to set privoxy to accept a ridiculously high amount of connections to even get the videos to load... I also tested setting up proxychain and it was much more reliable. I think what's causing my issues is this: https://github.com/mamantoha/http_proxy/issues/31
I think we should wait until this implementation can correctly use a connection pooler.
I'd say let's get this rolling for people whose proxy can accept insane amounts of connections, and upgrade to the pooling solution as soon as it becomes available. In cases where the proxy is weak (or reasonably limited) but it's ok to refuse excessive connections, tcpfwd-maxconc may help. I use that to tame npm.
I was able to mitigate my problems by using some more proxies and putting a HAProxy as a load balancer and connection pooler in between. DASH is still somewhat unreliable but everything else works perfectly.