scrapoxy icon indicating copy to clipboard operation
scrapoxy copied to clipboard
verified publisher icon fabienvauchelles

aws instances being created and destroyed on its own over and over

Open shairyar opened this issue 7 years ago • 1 comments

Hello,

I am using the latest version of Scrapoxy and I have proxy authentication in place. The moment I start sending requests via scrapoxy it shuts down the alive instance (after sending the first request successfully) and creates a new instance, shuts that down, creates a new one and goes on forever.

I cant seem to figure out why that is, I thought I was getting banned so I did an experiment by sending the request to another domain I own to see what happens, but the behaviour is same, the first request is a successful one and right after that it shuts down that instance and start creating and destroying aws instances for ever. This continues even if i am no longer sending requests via scrapoxy.

I have to shut down the scrapoxy so it stops creating the destroying the instances.

Here is what I am seeing in log but there is nothing in it that can tell me whats happening (as far as my knowledge goes).

0|scrapoxy  | 2018-09-06T12:33:43.194Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:33:44.869Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:33:44.872Z - debug: [Instance/i-07159ff2fb8f914e5] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:33:44.873Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:33:44.873Z - debug: [Instance/i-0c7588241c0a21983] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:33:44.874Z - debug: [Manager] adjustInstances: required:3 / actual:3
0|scrapoxy  | 2018-09-06T12:33:53.195Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:33:54.721Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:33:54.723Z - debug: [Instance/i-0d6ab38443872e690] changeAlive: true => false
0|scrapoxy  | 2018-09-06T12:33:54.724Z - debug: [Instance/i-0d6ab38443872e690] Instance is down. Crash timer starts
0|scrapoxy  | 2018-09-06T12:33:54.725Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:33:54.726Z - debug: [Instance/i-07159ff2fb8f914e5] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:33:54.727Z - debug: [Manager] adjustInstances: required:3 / actual:1
0|scrapoxy  | 2018-09-06T12:33:54.728Z - debug: [Manager] adjustInstances: add 2 instances
0|scrapoxy  | 2018-09-06T12:33:54.729Z - debug: [ProviderAWSEC2] createInstances: count=2
0|scrapoxy  | 2018-09-06T12:34:03.197Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:04.907Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:04.909Z - debug: [Instance/i-0821f7b91cc93bfdb] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:04.910Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:04.911Z - debug: [Instance/i-0f60bbdaa1abe2290] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:04.912Z - debug: [Instance/i-0c7588241c0a21983] checkAlive every 20000 secs
0|scrapoxy  | 2018-09-06T12:34:04.913Z - debug: [Instance/i-0c7588241c0a21983] autorestart in 35493934 secs
0|scrapoxy  | 2018-09-06T12:34:04.914Z - debug: [Manager] adjustInstances: required:3 / actual:3
0|scrapoxy  | 2018-09-06T12:34:13.199Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:14.796Z - debug: [Instance/i-0821f7b91cc93bfdb] checkAlive every 20000 secs
0|scrapoxy  | 2018-09-06T12:34:14.798Z - debug: [Instance/i-0821f7b91cc93bfdb] autorestart in 16211008 secs
0|scrapoxy  | 2018-09-06T12:34:14.799Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:14.804Z - debug: [Instance/i-0c7588241c0a21983] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:14.806Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:14.807Z - debug: [Instance/i-0f60bbdaa1abe2290] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:14.808Z - debug: [Manager] adjustInstances: required:3 / actual:1
0|scrapoxy  | 2018-09-06T12:34:14.809Z - debug: [Manager] adjustInstances: add 2 instances
0|scrapoxy  | 2018-09-06T12:34:14.809Z - debug: [ProviderAWSEC2] createInstances: count=2
0|scrapoxy  | 2018-09-06T12:34:23.202Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:24.942Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:24.945Z - debug: [Instance/i-05aa9e26e9dd4af37] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:24.952Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:24.953Z - debug: [Instance/i-025ffdb8c3f355fcc] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:24.954Z - debug: [Manager] adjustInstances: required:3 / actual:3
0|scrapoxy  | 2018-09-06T12:34:33.205Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:34.794Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:34.795Z - debug: [Instance/i-05aa9e26e9dd4af37] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:34.796Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:34.797Z - debug: [Instance/i-025ffdb8c3f355fcc] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:34.797Z - debug: [Manager] adjustInstances: required:3 / actual:1
0|scrapoxy  | 2018-09-06T12:34:34.798Z - debug: [Manager] adjustInstances: add 2 instances
0|scrapoxy  | 2018-09-06T12:34:34.798Z - debug: [ProviderAWSEC2] createInstances: count=2
0|scrapoxy  | 2018-09-06T12:34:34.802Z - debug: [Instance/i-0821f7b91cc93bfdb] checkAlive: false / -
0|scrapoxy  | 2018-09-06T12:34:34.802Z - debug: [Pinger] ping: hostname=13.238.185.237 / port=3128
0|scrapoxy  | 2018-09-06T12:34:35.007Z - debug: [Instance/i-0821f7b91cc93bfdb] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:43.207Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:44.792Z - debug: [Manager] checkInstances: add: i-0d0191e160a96b6a7
0|scrapoxy  | 2018-09-06T12:34:44.794Z - debug: [Instance/i-0d0191e160a96b6a7] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:44.795Z - debug: [Manager] checkInstances: add: i-0f505543b16b83535
0|scrapoxy  | 2018-09-06T12:34:44.796Z - debug: [Instance/i-0f505543b16b83535] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:44.797Z - debug: [Manager] adjustInstances: required:3 / actual:3
0|scrapoxy  | 2018-09-06T12:34:53.209Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:34:54.844Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:34:54.845Z - debug: [Instance/i-0821f7b91cc93bfdb] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:54.846Z - debug: [Manager] checkInstances: remove: i-0d0191e160a96b6a7
0|scrapoxy  | 2018-09-06T12:34:54.851Z - debug: [Instance/i-0d0191e160a96b6a7] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:34:54.852Z - debug: [Manager] adjustInstances: required:3 / actual:1
0|scrapoxy  | 2018-09-06T12:34:54.852Z - debug: [Manager] adjustInstances: add 2 instances
0|scrapoxy  | 2018-09-06T12:34:54.852Z - debug: [ProviderAWSEC2] createInstances: count=2
0|scrapoxy  | 2018-09-06T12:35:03.210Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:35:04.883Z - debug: [Instance/i-0f505543b16b83535] checkAlive every 20000 secs
0|scrapoxy  | 2018-09-06T12:35:04.884Z - debug: [Instance/i-0f505543b16b83535] autorestart in 19699566 secs
0|scrapoxy  | 2018-09-06T12:35:04.885Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:35:04.886Z - debug: [Instance/i-07219e8a98995e212] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:35:04.887Z - debug: [Manager] checkInstances: add: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:35:04.887Z - debug: [Instance/i-05c592796ecdf86aa] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:35:04.889Z - debug: [Manager] adjustInstances: required:3 / actual:3
0|scrapoxy  | 2018-09-06T12:35:13.213Z - debug: [Manager] checkInstances
0|scrapoxy  | 2018-09-06T12:35:14.829Z - debug: [Instance/i-07219e8a98995e212] checkAlive every 20000 secs
0|scrapoxy  | 2018-09-06T12:35:14.831Z - debug: [Instance/i-07219e8a98995e212] autorestart in 33868750 secs
0|scrapoxy  | 2018-09-06T12:35:14.832Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:35:14.833Z - debug: [Instance/i-0f505543b16b83535] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:35:14.834Z - debug: [Manager] checkInstances: remove: [email protected]:3128
0|scrapoxy  | 2018-09-06T12:35:14.835Z - debug: [Instance/i-05c592796ecdf86aa] changeAlive: false => false
0|scrapoxy  | 2018-09-06T12:35:14.836Z - debug: [Manager] adjustInstances: required:3 / actual:1
0|scrapoxy  | 2018-09-06T12:35:14.836Z - debug: [Manager] adjustInstances: add 2 instances
0|scrapoxy  | 2018-09-06T12:35:14.837Z - debug: [ProviderAWSEC2] createInstances: count=2

I will really appreciate a quick response on this one as I am not able to use scrapoxy at all.

shairyar avatar Sep 06 '18 12:09 shairyar

I have had something similar happen that after a certain time it will start killing and starting the nodes, also the scrapoxy script itself seems to be dying a bunch. What usually helps is to kill the server and start it up again. Not sure why it does that tho.

MaZZly avatar Dec 11 '18 11:12 MaZZly

connectivity issue. solved in 4.0.0

scrapoxy avatar Oct 04 '23 20:10 scrapoxy

Hey there! 🌟 Exciting news! Scrapoxy 4 is ready to rock 🚀. Check it out at Scrapoxy.io (explore the "get started" guide, deployment documentation, and more 📚). I can't wait to hear your feedback on this new version! Send me your coolest screenshots with as many proxies as possible! 📸💻 Join the Discord community if you have any questions or just want to chat. You can also open a GitHub issue for any bug or feature request 🐞✨. See you soon! 😎 Fabien

scrapoxy avatar Dec 12 '23 22:12 scrapoxy