issues due to sites more than 10.

[ankurghai]

Hey. my server is crashing every now and then either says Try running less websites or contact Damm998. and the second error at times i get is too many tasks for node.

i had 30 websites in the monitor. then tried creating three versions of the monitor in different directories to run 10 sites each. still got the errors after some time. what am I doing wrong or what would be the best way to setup multiple monitors on my site without them clashing with each other? i have around 30 to 40 sites to monitor.

Thanks.

[Dam998]

If you run multiple instances you need to change the db name in the config.json because if you didn't all instance work over the same db, so the monitors crash

in config.json you have: "mongodb://127.0.0.1:27017/shopify" in this string change the word 'shopify' for the different instances, no need what do you write, you can write what you want

if you run 3 instances, for example, you can write: "mongodb://127.0.0.1:27017/shopify1" "mongodb://127.0.0.1:27017/shopify2" "mongodb://127.0.0.1:27017/shopify3" in the different config.json files

[ankurghai]

so i did the setup as you said. now i am getting this error today. And i am getting it on a single instance as well

C:\shopify-monitor\src\classes\Discord.js:16 global.config.webhook_url.forEach(x => { ^

TypeError: global.config.webhook_url.forEach is not a function at Object. (C:\shopify-monitor\src\classes\Discord.js:16:27) at Module._compile (internal/modules/cjs/loader.js:1158:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10) at Module.load (internal/modules/cjs/loader.js:1002:32) at Function.Module._load (internal/modules/cjs/loader.js:901:14) at Module.require (internal/modules/cjs/loader.js:1044:19) at require (internal/modules/cjs/helpers.js:77:18) at Object. (C:\shopify-monitor\src\classes\Task.js:4:17) at Module._compile (internal/modules/cjs/loader.js:1158:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10) at Module.load (internal/modules/cjs/loader.js:1002:32) at Function.Module._load (internal/modules/cjs/loader.js:901:14) at Module.require (internal/modules/cjs/loader.js:1044:19) at require (internal/modules/cjs/helpers.js:77:18) at Object. (C:\shopify-monitor\index.js:6:14) at Module._compile (internal/modules/cjs/loader.js:1158:30)

[Dam998]

maybe because in your config.json file you didn't insert the webhook_url in an array, like I did in the example

[ankurghai]

Done. So they are up and running now. I'll update here in a few hours. Although i hope they should be fine now. :) Thanks btw

[Dam998]

you're welcome

[ankurghai]

just bought 5 dc proxies and started the instances on them. working like a charm. gonna go lower than 30 sec now. btw, what's this error linked to?

[ERROR ] cncpts.com has an high level of protection from monitors, please notify Dam998 by opening an issue on github

[ankurghai]

why is this happening a lot in between buddy? i am not sure I understand why this error occurs.

code: 'ERR_SOCKET_CLOSED', May 11 09:57:32 streetbud-monitor node[16748]: config: { May 11 09:57:32 streetbud-monitor node[16748]: url: 'https://wishatl.com/products.json', May 11 09:57:32 streetbud-monitor node[16748]: method: 'get', May 11 09:57:32 streetbud-monitor node[16748]: headers: { May 11 09:57:32 streetbud-monitor node[16748]: Accept: 'application/json, text/plain, /', May 11 09:57:32 streetbud-monitor node[16748]: 'User-Agent': 'axios/0.19.2' May 11 09:57:32 streetbud-monitor node[16748]: }, May 11 09:57:32 streetbud-monitor node[16748]: transformRequest: [ [Function: transformRequest] ], May 11 09:57:32 streetbud-monitor node[16748]: transformResponse: [ [Function: transformResponse] ], May 11 09:57:32 streetbud-monitor node[16748]: timeout: 0, May 11 09:57:32 streetbud-monitor node[16748]: adapter: [Function: httpAdapter], May 11 09:57:32 streetbud-monitor node[16748]: xsrfCookieName: 'XSRF-TOKEN', May 11 09:57:32 streetbud-monitor node[16748]: xsrfHeaderName: 'X-XSRF-TOKEN', May 11 09:57:32 streetbud-monitor node[16748]: maxContentLength: -1, May 11 09:57:32 streetbud-monitor node[16748]: validateStatus: [Function: validateStatus], May 11 09:57:32 streetbud-monitor node[16748]: httpsAgent: HttpsProxyAgent { May 11 09:57:32 streetbud-monitor node[16748]: _events: [Object: null prototype] {}, May 11 09:57:32 streetbud-monitor node[16748]: _eventsCount: 0, May 11 09:57:32 streetbud-monitor node[16748]: _maxListeners: undefined, May 11 09:57:32 streetbud-monitor node[16748]: timeout: null, May 11 09:57:32 streetbud-monitor node[16748]: maxFreeSockets: 1, May 11 09:57:32 streetbud-monitor node[16748]: maxSockets: 1, May 11 09:57:32 streetbud-monitor node[16748]: sockets: {}, May 11 09:57:32 streetbud-monitor node[16748]: requests: {}, May 11 09:57:32 streetbud-monitor node[16748]: secureProxy: false, May 11 09:57:32 streetbud-monitor node[16748]: proxy: [Object], May 11 09:57:32 streetbud-monitor node[16748]: promisifiedCallback: [Function: callback], May 11 09:57:32 streetbud-monitor node[16748]: [Symbol(kCapture)]: false

[Dam998]

I confirm you that cncpts.com have a protection from monitor, I'll check in the next days how can I pass it but maybe I need to create another monitor specific for that site, I don't know, I'll check it

[Dam998]

for the error: do you use proxies?

[ankurghai]

for the error: do you use proxies?

yes i do. have 5 private proxies filled in them currently.

[Dam998]

but the error happen all time you start the monitor? after, more or less, 1 minute?

[ankurghai]

but the error happen all time you start the monitor? after, more or less, 1 minute?

i get this error most of the times throughout my instances, but it's not specific which site it's happening on. it's almost on everyone at some point or another.. doesnt happen straightaway but starts coming up in atleast one of the instance after the first 2-3 minutes. currently i have 4 instances running, all filled with these errors. i am still getting some restock updates but not sure if they are all the stores or just a few which are not crashing.

[Dam998]

it's seems to be an error of the request with a proxy did you write the proxies well? example http://asdf:[email protected]:8000

where: asdf is the username qwerty is the password

[ankurghai]

ohh i think i know what the problem is. so, my proxies have 3 concurrent connection support. so maybe they are crashing coz of that. it just has to reset when a 4th connection happens.

just to be clear. if i have 4 instances running up, and i am using 8 proxies which support upto 3 connections at a time. how should i place them between my instances to make sure they dont clash.

also do i need to change the timeout in the index.js to make tasks reload a second late?

[ankurghai]

or should i get more proxies and fill them up? how many would you reckon should not cause any issue and how low can i go with the requestTiming then?

[Dam998]

Hi, about your last error seems that proxy socket close too early, I reproduce it on my computer this is not a problem of the monitor because with only pc (without proxy) it's working well

I reproduce it with 4 istance of monitor with the same proxies running

I would like to inform you that I'm working on find a solution on this problem, hope to find it fast I'll update you here when I have news

[Dam998]

do you still have this problem?

[ankurghai]

yepp. like it's not affecting the monitor in the long run as eventually they all connect back. i created a discord bot for monitoring the status of each site inside my monitor instances. so most of the times all sites are connected. only get to see this when I open the JournalCTL of my server.

[asdat3]

hey, I am sorry I can´t help you, but I am trying since 3 days now to run the monitor 2 times on my windows laptop and I am not able to... I tried nodeenv and got nowhere. Please help

[ankurghai]

hey, I am sorry I can´t help you, but I am trying since 3 days now to run the monitor 2 times on my windows laptop and I am not able to... I tried nodeenv and got nowhere.

Please help

Make sure you change mongodb name in the config.json file like “ipaddress:port/shopify” and “ipaddress:port/shopify2” keeping it different for both of the instances of monitor.. other than that just keep them in two separate folders. Npm install in both of them.. then npm start in both of the directories.

[Dam998]

yes, you need to change the mongo db name

however @ankurghai do you have the same problem? the ERR_SOCKET_CLOSED? are you using proxy?

[asdat3]

Thank you so much <3

[ankurghai]

yes, you need to change the mongo db name

however @ankurghai do you have the same problem? the ERR_SOCKET_CLOSED?

are you using proxy?

Yes brother. Still the same. Please read my message two posts above. Tomorrow i’ll get sme more proxies... and i’ll try to put unique proxies in each monitor... lets see if they work without the error then.. and also, does the monitor switch proxy when there is a ban or it circulates proxy with each time a site is accessed?

[Dam998]

try to remove ?limit=250 in task.js, let me know if works