pystemon
pystemon copied to clipboard
Add proxy use per site
Hello,
For some needs I've implemented some changes on pystemon in order to be able to use a proxy only for some site. It is now possible to add use-proxy directive to site in order to use random proxy only on the select site
Now the ProxyList is setted if proxy.random: yes
or if one enabled site have use-proxy: yes
in his configuration.
Example :
pastebin.gr:
enable: no
archive-url: 'http://pastebin.gr/archive'
archive-regex: '<td><a href="(\d+)" title='
download-url: 'http://pastebin.gr/paste.php?download&id={id}'
throttling: 5000
use-proxy: yes
If proxy.random: yes
proxy will still be used for all sites
I have also disabled sslVerify in order to be able to proxify https request to http servers and added 'https' for proxy in session.
The code is maybe not clean at all or far away from your guidelines. Do not hesitate to tell me if something is bad :)
Cheers !
Hi, Why this feature is not merged ? It is very usefull with pastebin.com pro for instance which need a specific proxy associated with a whitelisted IP.
Moreover it fixes an issue on calling https request through proxy in ua.py l. 196, by adding https to session proxy:
if random_proxy:
session.proxies = {'http': random_proxy, 'https': random_proxy}