shopify-monitor
shopify-monitor copied to clipboard
dzt
Connection error to every site even with 1000000 Conn delay
I'm getting a connection error to every site. My settings are: "pollTimeMs": 1000000, "proxies": false, "slackBot": { "active": true, "token": "xxxxxxxxxxxxxxxxx", "channel": "monitor", "settings": { "username": "shopifymonitor", "icon_url": "https://i.imgur.com/E0BubGr.png" } }, "discord": { "active": false, "webhook_url": "url here" }, "twitter": { "active": true, "encodeImages": false, "consumer_key": "xxxxxx", "consumer_secret": "xxxxxxxxxxx", "access_token": "xxxxxxxx", "access_token_secret": "xxxxxxxxxx" }, "database": { "client": "sqlite3", "connection": { "filename": "./monitor-database.sqlite3" }, "acquireConnectionTimeout": 1000000, "useNullAsDefault": true
I'm having the same problem. I've tried various poll times and also different variations of sites (kith.com vs kith's sitemap) but to no avail.
Exactly I was having the same problem and ended up getting blocked by the developer because they didn't understand how to develop., I showed the developer a screenshot of my config where acquire connection was set to 60,000 and I think he ignored it because a couple hours later he commented on the closed issue saying the solution is to change the config to 60,000. https://github.com/dzt/shopify-monitor/issues/72 But the truth is I did some research. Changing the config to 60000 makes NO change to the issue at all.
See I researched the database library they are using. It's knex. It says in the documentation that the default value for acquireConnectionTimeout is already 60000 from the start. http://knexjs.org/#Installation-acquireConnectionTimeout "acquireConnectionTimeout defaults to 60000ms". So the real problem in shopify-monitor is that the devs did not properly program the way shopify uses the database. I couldn't figure out because I'm not a javascript developer but the error does say "The pool is probably full. Are you missing a .transacting(trx) call?" A database pool works by having a set number of connections and the code must temporarily use a connection then put it back in the pool. If it's saying the pool is full, that means the code in shopify is opening database connections without ever closing them. Also I searched the entire code for something that looks like " .transacting(trx)" and I couldn't find anything. Something is clearly wrong and the knex is not being used properly.
No transacting(trx) errors here, from what I can see. Only the connection errors. Hopefully the dev(s) will see this soon.
Yeah i think with all the updating the devs are doing, it's not a finished product just yet. Everything loads up and works besides the connection errors.
What I've done to get things going for now was dropped down to v2.0. Hopefully a resolution to the connection errors will come out later.
Connection errors are exactly what they seem. They mean that there was a problem connecting to the website. This means one of a few things:
- you have a problem with your internet
- you have a problem with proxies
- the site is unavailable(down)
Parsing errors, however, can mean a few things. Either the response data is malformed(failed to retrieve data correctly), or you are banned.
A transaction call, as I'm told, is due to not enough resources(memory?)
If any of you have any more questions, please feel free to DM me @cgalt23 on twitter.
ajshaffer skycorrigan u guys got anything that i can contact u guys with? discord? twitter?
Follow my Twitter @bigburrtha
On Nov 23, 2017, at 3:16 AM, Johnwall719 <[email protected]mailto:[email protected]> wrote:
ajshaffer skycorrigan u guys got anything that i can contact u guys with? discord? twitter?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/dzt/shopify-monitor/issues/90#issuecomment-346553184, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AS7Q4_qFqviMrI7QTZH0Lq2CTOZ3CDJqks5s5SnHgaJpZM4QmWeH.
Make sure you clean out the proxy file. I found even with proxy set to false, the program was loading up the text in proxy file and using it to connect. All was well after clearing out the proxy.txt file saving and restarting.
@bennt1 Thank you for the tip! It was exactly the fix that I needed. I just wish that I knew of it earlier as I checked out every file in the program, except this one, for something that seemed out of place.
glad I could help, I was driven to the edge too!
On Fri, Nov 24, 2017 at 11:21 PM, skycorrigan [email protected] wrote:
@bennt1 https://github.com/bennt1 Thank you for the tip! It was exactly the fix that I needed. I just wish that I knew of it earlier as I checked out every file in the program, except this one, for something that seemed out of place.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dzt/shopify-monitor/issues/90#issuecomment-346924034, or mute the thread https://github.com/notifications/unsubscribe-auth/AV9AgMeNIYpCF6TJgIYhqUHqzt7SsoiNks5s57_wgaJpZM4QmWeH .
My config:
"database": { "client": "sqlite3", "connection": { "filename": "./monitor-database.sqlite3" }, "acquireConnectionTimeout": 700000, "useNullAsDefault": true },
Errors:
[Sun Nov 19 2017 19:22:39] [LOG] err { TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call ? at Timeout._onTimeout (\shopify\node_modules\kn ex\lib\client.js:287:18) at ontimeout (timers.js:386:11) at tryOnTimeout (timers.js:250:5) at Timer.listOnTimeout (timers.js:214:5) sql: undefined, bindings: undefined } I was instructioned by this software's developer to increase acquireConnectionTimeout but it didn't help solve this error I'm receiving. does anyone know how the error can be fixed?
I'm no export at javascript or knex but perhaps the program isn't closing its database connections and the pool is becoming full of opened connections?
@Johnwall719 Could you post your whole config? That way I can compare it against mine. Also, did you clear out your proxies.txt file (if you're not using proxies) as mentioned by @bennt1?
I'm no export at javascript or knex but perhaps the program isn't closing its database connections and the pool is becoming full of opened connections?
Just an idea,
Maybe try adding the debug switch to your config.json ? It just might add some more detailed error messages to the log. It looks like your not binding at all.
"database": { "client": "sqlite3", "debug": "true", "connection": { "filename": "./monitor-database.sqlite3" }, "acquireConnectionTimeout":600000, "useNullAsDefault": true },
also check your sqlite install with something like npm-check. it will show the installed module versions.
$ npm install -g npm-check
https://www.npmjs.com/package/npm-check#useUse
$ npm-check
just my 2cents.
I would be happy to teamview in to your machine if you want to screen share.
-jason
On Wed, Dec 27, 2017 at 7:41 PM, Johnwall719 [email protected] wrote:
My config:
"database": { "client": "sqlite3", "connection": { "filename": "./monitor-database.sqlite3" }, "acquireConnectionTimeout": 700000, "useNullAsDefault": true },
Errors:
[Sun Nov 19 2017 19:22:39] [LOG] err { TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call ? at Timeout._onTimeout (\shopify\node_modules\kn ex\lib\client.js:287:18) at ontimeout (timers.js:386:11) at tryOnTimeout (timers.js:250:5) at Timer.listOnTimeout (timers.js:214:5) sql: undefined, bindings: undefined } I was instructioned by this software's developer to increase acquireConnectionTimeout but it didn't help solve this error I'm receiving. does anyone know how the error can be fixed?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dzt/shopify-monitor/issues/90#issuecomment-354222969, or mute the thread https://github.com/notifications/unsubscribe-auth/AV9AgKpTTZqqUJl8hRN7qCCVNcKAXueJks5tEw4AgaJpZM4QmWeH .
@Johnwall719 No contacts on social media. I just have my email.
Couple of questions:
- What's your OS?
- Have you tried the install in a virtual machine (VM)?
After trying @bennt1 suggestions, I'd recommend doing the install in a VM so that you have a fresh baseline w/o any possible conflicting software.
im having the same problem guys and there no text in my proxy.txts other than what the dev wrong as an example
ACGDeptordsiMac:shopify-monitor acgdeptford$ npm start
[email protected] start /Users/acgdeptford/Desktop/shopify-monitor babel-node index.js
[Fri Dec 29 2017 16:19:53] [LOG] [ 'host:port:username:password', 'or', 'host:port', 'line by line' ] [Fri Dec 29 2017 16:19:53] [LOG] ------------------------------------------------ [Fri Dec 29 2017 16:19:53] [LOG] ShopifyMonitor v2 [Fri Dec 29 2017 16:19:53] [LOG] ------------------------------------------------ [Fri Dec 29 2017 16:19:53] [LOG] Developed by dzt [Fri Dec 29 2017 16:19:53] [LOG] ------------------------------------------------ [Fri Dec 29 2017 16:19:53] [LOG] config.json file has been loaded [Fri Dec 29 2017 16:19:54] [LOG] Starting Tasks... [Fri Dec 29 2017 16:19:54] [LOG] All tasks have been successfully Initialized [Fri Dec 29 2017 16:19:54] [LOG] ------------------------------------------------ [Fri Dec 29 2017 16:19:54] [LOG] Slack Bot Enabled [Fri Dec 29 2017 16:19:54] [LOG] Discord Bot Enabled [Fri Dec 29 2017 16:19:54] [LOG] ------------------------------------------------ [Fri Dec 29 2017 16:19:54] [LOG] Connection Error @ https://fuckingawesomestore.com, polling again in 300000ms [Fri Dec 29 2017 16:19:54] [LOG] Connection Error @ https://fuckthepopulation.com, polling again in 300000ms [Fri Dec 29 2017 16:19:54] [LOG] Connection Error @ https://shop.dertbag.us, polling again in 300000ms [Fri Dec 29 2017 16:19:54] [LOG] Connection Error @ https://store.illegalcivilization.com, polling again in 300000ms [Fri Dec 29 2017 16:19:54] [LOG] Connection Error @ https://offthehook.ca, polling again in 300000ms ect.....
this is my npm-check
ACGDeptordsiMac:shopify-monitor acgdeptford$ npm-check
from 😕 NOTUSED? Still using from? Depcheck did not find code similar to require('from') or import from 'from'. Check your code before removing as depcheck isn't able to foresee all ways dependencies can be used. Use --skip-unused to skip this check. To remove this package: npm uninstall --save from
import 😕 NOTUSED? Still using import? Depcheck did not find code similar to require('import') or import from 'import'. Check your code before removing as depcheck isn't able to foresee all ways dependencies can be used. Use --skip-unused to skip this check. To remove this package: npm uninstall --save import
lodash 😕 NOTUSED? Still using lodash? Depcheck did not find code similar to require('lodash') or import from 'lodash'. Check your code before removing as depcheck isn't able to foresee all ways dependencies can be used. Use --skip-unused to skip this check. To remove this package: npm uninstall --save lodash
sqlite3 😕 NOTUSED? Still using sqlite3? Depcheck did not find code similar to require('sqlite3') or import from 'sqlite3'. Check your code before removing as depcheck isn't able to foresee all ways dependencies can be used. Use --skip-unused to skip this check. To remove this package: npm uninstall --save sqlite3
Use npm-check -u for interactive update.
@trvpbo0om, the proxies.txt file should be empty if you're not using proxies. Meaning, delete the proxy example lines from the file. Save. Then restart.
@skycorrigan thanks. im kind of mad my problem wasn't more complicated lol
didnt even notice my first line was lol
[Fri Dec 29 2017 16:19:53] [LOG] [ 'host:port:username:password', 'or', 'host:port', 'line by line' ] [Fri Dec 29 2017 16:19:53] [LOG] ------------------------------------------------
[Sat Dec 30 2017 17:18:11] [LOG] [] [Sat Dec 30 2017 17:18:11] [LOG] ------------------------------------------------ [Sat Dec 30 2017 17:18:11] [LOG] ShopifyMonitor v2 [Sat Dec 30 2017 17:18:11] [LOG] ------------------------------------------------ [Sat Dec 30 2017 17:18:11] [LOG] Developed by dzt [Sat Dec 30 2017 17:18:11] [LOG] ------------------------------------------------ [Sat Dec 30 2017 17:18:11] [LOG] config.json file has been loaded [Sat Dec 30 2017 17:18:12] [LOG] Starting Tasks... [Sat Dec 30 2017 17:18:12] [LOG] All tasks have been successfully Initialized [Sat Dec 30 2017 17:18:12] [LOG] ------------------------------------------------ [Sat Dec 30 2017 17:18:12] [LOG] Slack Bot Enabled [Sat Dec 30 2017 17:18:12] [LOG] Discord Bot Enabled [Sat Dec 30 2017 17:18:12] [LOG] ------------------------------------------------ [Sat Dec 30 2017 17:18:12] [LOG] Initial Check (Successful): fuckthepopulation.com [Sat Dec 30 2017 17:18:13] [LOG] Initial Check (Successful): fuckingawesomestore.com [Sat Dec 30 2017 17:18:14] [LOG] Initial Check (Successful): store.illegalcivilization.com [Sat Dec 30 2017 17:18:15] [LOG] Initial Check (Successful): beatniconline.com [Sat Dec 30 2017 17:18:22] [LOG] Initial Check (Successful): sneakerpolitics.com [Sat Dec 30 2017 17:18:23] [LOG] Initial Check (Successful): shop.dertbag.us [Sat Dec 30 2017 17:18:27] [LOG] Initial Check (Successful): offthehook.ca [Sat Dec 30 2017 17:18:32] [LOG] Initial Check (Successful): commonwealth-ftgg.com [Sat Dec 30 2017 17:18:33] [LOG] Initial Check (Successful): trophyroomstore.com [Sat Dec 30 2017 17:18:38] [LOG] Initial Check (Successful): rockcitykicks.com [Sat Dec 30 2017 17:18:40] [LOG] Initial Check (Successful): rsvpgallery.com [Sat Dec 30 2017 17:18:45] [LOG] Initial Check (Successful): shop.extrabutterny.com [Sat Dec 30 2017 17:18:47] [LOG] Initial Check (Successful): saintalfred.com [Sat Dec 30 2017 17:18:47] [LOG] Initial Check (Successful): shop-usa.palaceskateboards.com [Sat Dec 30 2017 17:18:52] [LOG] Initial Check (Successful): lapstoneandhammer.com [Sat Dec 30 2017 17:18:56] [LOG] Initial Check (Successful): shop.exclucitylife.com [Sat Dec 30 2017 17:19:00] [LOG] Initial Check (Successful): wishatl.com [Sat Dec 30 2017 17:19:02] [LOG] Initial Check (Successful): xhibition.co [Sat Dec 30 2017 17:19:04] [LOG] Initial Check (Successful): notre-shop.com [Sat Dec 30 2017 17:19:12] [LOG] Initial Check (Successful): kith.com [Sat Dec 30 2017 17:19:14] [LOG] Initial Check (Successful): burnrubbersneakers.com [Sat Dec 30 2017 17:19:18] [LOG] Initial Check (Successful): socialstatuspgh.com [Sat Dec 30 2017 17:19:32] [LOG] Initial Check (Successful): shopnicekicks.com [Sat Dec 30 2017 17:19:49] [LOG] Initial Check (Successful): shop.bdgastore.com
please let me know what im doing wrong
can we get a update to fix the error we having ?
been running this all day an still nothing :-1:
We can. Pass me your TeamViewer ID and I'll connect in.
On Dec 30, 2017 at 5:15 PM, <Johnwall719 (mailto:[email protected])> wrote:
@Sky (https://github.com/sky) Corrigan can we teamview?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/dzt/shopify-monitor/issues/90#issuecomment-354571377), or mute the thread (https://github.com/notifications/unsubscribe-auth/AgT0-4AOHhZeQ91kVeIqyOozsk_4t-cMks5tFrXtgaJpZM4QmWeH).
@camron114 If your timezone is EST, then from your post it looks like you just started the program and it's running correctly. Do remember that the program will only provide output when new and/or restock items from your site and keyword list are detected. Give it time; the program works.
If you want to test it to see it in action, put some generic keywords (think Nike, Adidas, etc) in your config.json file, save, and restart.