requests-ip-rotator
requests-ip-rotator copied to clipboard
Ge0rg3
A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.
Hi guys. I just installed the solution and did all the configuration (AWS ID, Secret, etc). When I call session.get I get in error message "'ApiGateway' object has no attribute...
Hello sir, very nice module :) I am trying to make it nicer by speeding up closing and opening of gateways. Current implementation uses: `apis = awsclient.get_rest_apis()["items"]` // to get...
Hey man, Amazing module you have here and helped me a lot, I'm getting the following error on "some" requests, any idea how to fix it? **requests.exceptions.ConnectionError: HTTPSConnectionPool(host='', port=443): Pool...
Hey, I am back again:) I was happily parsing last few days away, blew past the free tier already, but I wanted to scale up my operation further to make...
Hey there, thanks a lot for this great lib! I noticed that it uses the REST api. Would it theoretically be possibe to use the HTTP Api, which is only...
I have looked into the REST API pricing, the prices seem to be between 3$ and 4$ for
So far this has been an awesome repo. Unfortunately, I am trying to do something perhaps a little outside of the scope and wanted to see if there was any...
Hi there? Nice to meet you. Here is a sample code, I copied from this github repo. from os import access import time import requests from requests_ip_rotator import ApiGateway, DEFAULT_REGIONS...
AWS IPs are working great for scraping, but I'd like to add redundancy. Are there any other services like Google Cloud that can achieve the same? Thanks!
Hi All - Great project! Just started testing this repo but it looks like CloudWatch stats are delayed. I'm wondering if anyone knows whether the API Gateway charges for failed...