stash icon indicating copy to clipboard operation
stash copied to clipboard

[Feature] SOCKS/Proxy support for scraping

Open EZarnosky opened this issue 5 years ago • 4 comments

Is your feature request related to a problem? Please describe. SOCKS/Proxy support

Describe the solution you'd like Enable the use of proxy servers for scraping of data

EZarnosky avatar Jul 27 '20 07:07 EZarnosky

Any process on this?

troyliu0105 avatar Jan 09 '22 03:01 troyliu0105

Go respects the HTTP_PROXY environment variable by default so this should be possible already.

e.g.

HTTP_PROXY=socks5://127.0.0.1:8080 ./stashapp

... or when using Docker:

docker run -e HTTP_PROXY=socks5://127.0.0.1:8080 stashapp/stash:development

Docs: https://pkg.go.dev/net/http#ProxyFromEnvironment

Of course this would affect all outbound connections not just scraping (e.g. Stashbox). You can use NO_PROXY to exclude hosts from the proxy.

joyov avatar Apr 17 '22 12:04 joyov

This seems to work for checking stashdb, but it doesn't seem to be obeyed on many scrapers ( via the 'Scrape With...' button or by clicking the download button next to a URL ). Are these scrapers implemented differently by stash? There simply is no request via a proxy when using many of the site-specific scrapers, or generic (TPD) scrapers.

michaelkrieger avatar Jun 24 '22 04:06 michaelkrieger

setting neither HTTP_PROXY nor HTTPS_PROXY work, but using python version scraper works

snowman avatar Nov 09 '22 08:11 snowman

add proxy: socks5://127.0.0.1:8080 in the config.yml solves my problem

zoubingwu avatar Nov 01 '23 06:11 zoubingwu