couch-to-postgres icon indicating copy to clipboard operation
couch-to-postgres copied to clipboard

Published 20 hours ago • verified publisher icon sysadminmike

works on workstation, not working on server

Open jmorganh opened this issue 3 years ago • 27 comments

I worked through getting this to work on my desktop with PG and couch on our server. When I copy the folder( couch-to-Postgres) to my server (same server as pg and couch) it will not sync.

I get this on the server. there is blue text that starts with "follow:" This text was green when I first ran it.

follow:stream http://administrator:[email protected]:5984/tcsoffice:debug } +9ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug JSON: '{"seq":"25-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmxqaGiRYpzCwFmal5KalpmXmoJHex4LkGRoAFL_oaZIQkyxNDNISjbBpi8LAJ9ZLRw","id":"wo::1010","changes":[{"rev":"2-779eb03dff0f507cbedf9dd44b5c1a24"}],"deleted":true,"doc":{"_id":"wo::1010","_rev":"2-779eb03dff0f507cbedf9dd44b5c1a24","_deleted":true}}' +19ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug Object: { follow:stream http://administrator:[email protected]:5984/tcsoffice:debug seq: '25-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmxqaGiRYpzCwFmal5KalpmXmoJHex4LkGRoAFL_oaZIQkyxNDNISjbBpi8LAJ9ZLRw', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug id: 'wo::1010', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug changes: [ { rev: '2-779eb03dff0f507cbedf9dd44b5c1a24' } ], follow:stream http://administrator:[email protected]:5984/tcsoffice:debug deleted: true, follow:stream http://administrator:[email protected]:5984/tcsoffice:debug doc: { follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _id: 'wo::1010', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _rev: '2-779eb03dff0f507cbedf9dd44b5c1a24', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _deleted: true follow:stream http://administrator:[email protected]:5984/tcsoffice:debug } follow:stream http://administrator:[email protected]:5984/tcsoffice:debug } +3ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug JSON: '{"seq":"27-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmxqaGiRYpzCwFmal5KalpmXmoJHex4LkGRoAFL_oaZIQ0yxNDNISjbBpi8LAJ-dLR4","id":"wo::12365","changes":[{"rev":"2-e0d0a9a121da8776801f59ccea7cc691"}],"deleted":true,"doc":{"_id":"wo::12365","_rev":"2-e0d0a9a121da8776801f59ccea7cc691","_deleted":true}}' +6ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug Object: { follow:stream http://administrator:[email protected]:5984/tcsoffice:debug seq: '27-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmxqaGiRYpzCwFmal5KalpmXmoJHex4LkGRoAFL_oaZIQ0yxNDNISjbBpi8LAJ-dLR4', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug id: 'wo::12365', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug changes: [ { rev: '2-e0d0a9a121da8776801f59ccea7cc691' } ], follow:stream http://administrator:[email protected]:5984/tcsoffice:debug deleted: true, follow:stream http://administrator:[email protected]:5984/tcsoffice:debug doc: { follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _id: 'wo::12365', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _rev: '2-e0d0a9a121da8776801f59ccea7cc691', follow:stream http://administrator:[email protected]:5984/tcsoffice:debug _deleted: true follow:stream http://administrator:[email protected]:5984/tcsoffice:debug } follow:stream http://administrator:[email protected]:5984/tcsoffice:debug } +13ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug emit: data +3ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Data from 2021-03-26T20:33:11.106Z +2ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Req 2021-03-26T20:33:11.106Z timeout=37500, inactivity=30000: http://192.168.0.12:5984/tcsoffice +11ms example: Starting checkpointer example: Checkpoint 1 is current next check in: 120 seconds follow:http://administrator:[email protected]:5984/tcsoffice:debug Req 2021-03-26T20:33:11.106Z made no changes for 30.002s +30s follow:http://administrator:[email protected]:5984/tcsoffice:debug Stop +1ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Destroying req 2021-03-26T20:33:11.106Z +4ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug destroy +1ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Checking database: http://192.168.0.12:5984/tcsoffice +4ms follow:stream http://administrator:[email protected]:5984/tcsoffice:debug write: { data: '', buf: '' } +8ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Confirmed database: http://192.168.0.12:5984/tcsoffice +7ms example: {"db_name":"tcsoffice","purge_seq":"0-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshIZ8KhNZEiqhyjKAgBm5Rxs","update_seq":"151-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshL78KhNZEiqByvizAIAfJkdAw","sizes":{"file":467308,"external":95,"active":25024},"props":{"partitioned":true},"doc_del_count":39,"doc_count":1,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1},"instance_start_time":"0"} follow:http://administrator:[email protected]:5984/tcsoffice:debug Feed query 2021-03-26T20:33:41.278Z: http://192.168.0.12:5984/tcsoffice/_changes?since=11-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmxqaGiRYpzCwFmal5KalpmXmoJHex4LkGRoAFL_oaZwQ0yxNDNISjbBpi8LAJ19LQ4&feed=continuous&heartbeat=30000&include_docs=true +2ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Remove feed from agent pool: 2021-03-26T20:33:41.278Z +9ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Good response: 2021-03-26T20:33:41.278Z +1ms follow:http://administrator:[email protected]:5984/tcsoffice:debug Req 2021

jmorganh avatar Mar 26 '21 20:03 jmorganh

Hi

What happens if you try to install with npm on the server rather than copying it?

npm -i couch-to-postgres

Also what version of couch are you using as I have not tested it with 2.x and 3.x and have only used it with 1.x

sysadminmike avatar Mar 27 '21 09:03 sysadminmike

I use this repo. If it helps, below is how I am using it.

The index.js is started on this line. https://github.com/dgleba/listo429c/blob/cd6c31489e127d87913008e952e7af406d52c39e/docker-compose.example.yml#L115

You can see in the docker-compose.example.yml it uses couch 2.3 and postgres 11

Please don't let this post stop anyone else from helping. I hope this helps rather than confuses things.

dgleba avatar Mar 27 '21 15:03 dgleba

Hi

What happens if you try to install with npm on the server rather than copying it?

npm -i couch-to-postgres

Also what version of couch are you using as I have not tested it with 2.x and 3.x and have only used it with 1.x

I am using couchdb 3.1 sync works great from desktop with couchdb on server.

on server When i run "npm install couch-to-postgres" i get same messages when i run ./bin/index.js

I tried to run "git clone..." on the server and i get : [email protected]: Permission denied (publickey).

any ideas?

Thanks

jmorganh avatar Mar 29 '21 16:03 jmorganh

So i have downloaded a new folder of couch-to-postgres from git ran "npm install" on the new folder. adjusted index.js and this is what i get when i start it up:

C:\Program Files\couch-to-postgres-master>node ./bin/index.js follow:http://administrator:[email protected]:5984/tcsmaster:debug Checking database: http://127.0.0.1:5984/tcsmaster +0ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Confirmed database: http://127.0.0.1:5984/tcsmaster +24ms couch_import: {"db_name":"tcsmaster","purge_seq":"0-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshIZ8KhNZEiqhyjKAgBm5Rxs","update_seq":"7-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshLZ8ahNZEiqBytiyAIAZ_Yccw","sizes":{"file":45413,"external":58,"active":1385},"props":{"partitioned":true},"doc_del_count":1,"doc_count":2,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1},"instance_start_time":"0"} follow:http://administrator:[email protected]:5984/tcsmaster:debug Feed query 2021-03-29T16:53:36.213Z: http://127.0.0.1:5984/tcsmaster/_changes?since=0&feed=continuous&heartbeat=30000&include_docs=true +6ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Remove feed from agent pool: 2021-03-29T16:53:36.213Z +6ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Good response: 2021-03-29T16:53:36.213Z +2ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Req 2021-03-29T16:53:36.213Z timeout=37500, inactivity=30000: http://127.0.0.1:5984/tcsmaster +2ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug write: { data: '{"seq":"3-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaYwg00xsTQ1MzE2x6YvCwCSEyyq","id":"wo::12345","changes":[{"rev":"2-228d1dd4a2304550a476c91ac00db8cc"}],"deleted":true,"doc":{"_id":"wo::12345","_rev":"2-228d1dd4a2304550a476c91ac00db8cc","_deleted":true}}\n' + '{"seq":"6-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawgU0xsTQ1MzE2x6YvCwCSeSyt","id":"test:cb8ecd3c0090bd22c424ea8463000e7e","changes":[{"rev":"1-cab649096edff913e3bdef45519d31fd"}],"doc":{"_id":"test:cb8ecd3c0090bd22c424ea8463000e7e","_rev":"1-cab649096edff913e3bdef45519d31fd","name":"plenty of money"}}\n' + '{"seq":"7-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawg00xsTQ1MzE2x6YvCwCSmyyu","id":"test::3a8dc66a83658218fa78329a7302811a","changes":[{"rev":"4-4658ba72373d58829fef7109b37d55d7"}],"doc":{"_id":"test::3a8dc66a83658218fa78329a7302811a","_rev":"4-4658ba72373d58829fef7109b37d55d7","name":"lots of money money"}}\n', buf: '{"seq":"3-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaYwg00xsTQ1MzE2x6YvCwCSEyyq","id":"wo::12345","changes":[{"rev":"2-228d1dd4a2304550a476c91ac00db8cc"}],"deleted":true,"doc":{"_id":"wo::12345","_rev":"2-228d1dd4a2304550a476c91ac00db8cc","_deleted":true}}\n' + '{"seq":"6-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawgU0xsTQ1MzE2x6YvCwCSeSyt","id":"test:cb8ecd3c0090bd22c424ea8463000e7e","changes":[{"rev":"1-cab649096edff913e3bdef45519d31fd"}],"doc":{"_id":"test:cb8ecd3c0090bd22c424ea8463000e7e","_rev":"1-cab649096edff913e3bdef45519d31fd","name":"plenty of money"}}\n' + '{"seq":"7-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawg00xsTQ1MzE2x6YvCwCSmyyu","id":"test::3a8dc66a83658218fa78329a7302811a","changes":[{"rev":"4-4658ba72373d58829fef7109b37d55d7"}],"doc":{"_id":"test::3a8dc66a83658218fa78329a7302811a","_rev":"4-4658ba72373d58829fef7109b37d55d7","name":"lots of money money"}}\n' } +0ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug JSON: '{"seq":"3-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaYwg00xsTQ1MzE2x6YvCwCSEyyq","id":"wo::12345","changes":[{"rev":"2-228d1dd4a2304550a476c91ac00db8cc"}],"deleted":true,"doc":{"_id":"wo::12345","_rev":"2-228d1dd4a2304550a476c91ac00db8cc","_deleted":true}}' +2ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug Object: { seq: '3-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaYwg00xsTQ1MzE2x6YvCwCSEyyq', id: 'wo::12345', changes: [ { rev: '2-228d1dd4a2304550a476c91ac00db8cc' } ], deleted: true, doc: { _id: 'wo::12345', _rev: '2-228d1dd4a2304550a476c91ac00db8cc', _deleted: true } } +5ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug JSON: '{"seq":"6-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawgU0xsTQ1MzE2x6YvCwCSeSyt","id":"test:cb8ecd3c0090bd22c424ea8463000e7e","changes":[{"rev":"1-cab649096edff913e3bdef45519d31fd"}],"doc":{"_id":"test:cb8ecd3c0090bd22c424ea8463000e7e","_rev":"1-cab649096edff913e3bdef45519d31fd","name":"plenty of money"}}' +0ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug Object: { seq: '6-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawgU0xsTQ1MzE2x6YvCwCSeSyt', id: 'test:cb8ecd3c0090bd22c424ea8463000e7e', changes: [ { rev: '1-cab649096edff913e3bdef45519d31fd' } ], doc: { _id: 'test:cb8ecd3c0090bd22c424ea8463000e7e', _rev: '1-cab649096edff913e3bdef45519d31fd', name: 'plenty of money' } } +2ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug JSON: '{"seq":"7-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawg00xsTQ1MzE2x6YvCwCSmyyu","id":"test::3a8dc66a83658218fa78329a7302811a","changes":[{"rev":"4-4658ba72373d58829fef7109b37d55d7"}],"doc":{"_id":"test::3a8dc66a83658218fa78329a7302811a","_rev":"4-4658ba72373d58829fef7109b37d55d7","name":"lots of money money"}}' +1ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug Object: { seq: '7-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaawg00xsTQ1MzE2x6YvCwCSmyyu', id: 'test::3a8dc66a83658218fa78329a7302811a', changes: [ { rev: '4-4658ba72373d58829fef7109b37d55d7' } ], doc: { _id: 'test::3a8dc66a83658218fa78329a7302811a', _rev: '4-4658ba72373d58829fef7109b37d55d7', name: 'lots of money money' } } +2ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug emit: data +2ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Data from 2021-03-29T16:53:36.213Z +21ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Req 2021-03-29T16:53:36.213Z timeout=37500, inactivity=30000: http://127.0.0.1:5984/tcsmaster +7ms couch_import: Starting checkpointer couch_import: Checkpoint 1 is current next check in: 120 seconds

jmorganh avatar Mar 29 '21 17:03 jmorganh

When is is running (and not syncing) I get this about every 30sec:

follow:http://administrator:[email protected]:5984/tcsmaster:debug Req 2021-03-29T17:08:04.083Z made no changes for 30s +30s follow:http://administrator:[email protected]:5984/tcsmaster:debug Stop +0ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Destroying req 2021-03-29T17:08:04.083Z +2ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug destroy +30s follow:http://administrator:[email protected]:5984/tcsmaster:debug Checking database: http://127.0.0.1:5984/tcsmaster +3ms follow:stream http://administrator:[email protected]:5984/tcsmaster:debug write: { data: '', buf: '' } +3ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Confirmed database: http://127.0.0.1:5984/tcsmaster +4ms couch_import: {"db_name":"tcsmaster","purge_seq":"0-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshIZ8KhNZEiqhyjKAgBm5Rxs","update_seq":"8-g1AAAABXeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUnlsQBJhgYg9R8IshI58KhNZEiqBytiyAIAaB0cdA","sizes":{"file":49509,"external":77,"active":1562},"props":{"partitioned":true},"doc_del_count":1,"doc_count":3,"disk_format_version":8,"compact_running":false,"cluster":{"q":2,"n":1,"w":1,"r":1},"instance_start_time":"0"} follow:http://administrator:[email protected]:5984/tcsmaster:debug Feed query 2021-03-29T17:08:34.109Z: http://127.0.0.1:5984/tcsmaster/_changes?since=3-g1AAAACTeJzLYWBgYMpgTmEQTM4vTc5ISXLIyU9OzMnILy7JAUklMiTV____PyuDOZEhFyjAbmRoaGmYYpDCwFmal5KalpmXmoJHex4LkGRoAFL_oaYwg00xsTQ1MzE2x6YvCwCSEyyq&feed=continuous&heartbeat=30000&include_docs=true +5ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Remove feed from agent pool: 2021-03-29T17:08:34.109Z +6ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Good response: 2021-03-29T17:08:34.109Z +1ms follow:http://administrator:[email protected]:5984/tcsmaster:debug Req 2021-03-29T17:08:34.109Z timeout=37500, inactivity=30000: http://127.0.0.1:5984/tcsmaster +1ms

jmorganh avatar Mar 29 '21 17:03 jmorganh

It does look like its running but it looks like the following of the couch changes feed is timeing out for some reason - Im not really sure what is causing this and perhaps an issue with the "follow" dependency and the version of couch your using.

sysadminmike avatar Mar 30 '21 08:03 sysadminmike

One thing to try is using the daemon.js instead of the index.js - the index.js one was just an example of how to use the library and daemon.js is probably better to work with.

sysadminmike avatar Mar 30 '21 08:03 sysadminmike

I was wondering if it works on a machine that is not the database machine but I do not want it running on a desktop, maybe have it run on a server next to the database server. ( we have multiple VM servers in the network). I will also try the daemon.js and let you know. Thanks

jmorganh avatar Mar 30 '21 16:03 jmorganh

It may be worth just trying the Follow npm by itself with a simple script which just prints stuff on the console https://www.npmjs.com/package/follow - perhaps just see what happens with their example and couch 3:

var follow = require('follow'); follow("https://example.iriscouch.com/boogie", function(error, change) { if(!error) { console.log("Got change number " + change.seq + ": " + change.id); } })

sysadminmike avatar Mar 30 '21 17:03 sysadminmike

When i run daemon.js after setting config I get :

PG_WATCHDOG: reconnecting

on my desktop I get:

PG_WATCHDOG: OK WATCHDOG: Started WATCHDOG: Checking example WATCHDOG: example ok PG_WATCHDOG: OK

The only thing i change between the desktop and server is the address. 192.###.#.## to 127.0.0.1

I have tried both

jmorganh avatar Mar 30 '21 17:03 jmorganh

Ok to me thats pointing towards a setup config issue with postgres - double check the user/password and also postgres config allows connection - maybe check postgres logs for any clue

sysadminmike avatar Mar 30 '21 17:03 sysadminmike

yes, I do not see it as a session in PG. I do see my desktop as a session.

I would assume the pass would the same as the one that works.

jmorganh avatar Mar 30 '21 21:03 jmorganh

I have installed postbird on a neighboring VM server and connects just fine to PG. On this same server I cannot get couch-to-postgres to work.

Any more ideas?

Thanks Morgan

jmorganh avatar Mar 30 '21 21:03 jmorganh

Perhaps a permission issue and localhost/127.0.0.1 is missing from pg_hba.conf for the db your using? Postgres logs may be able to point in the right direction.

sysadminmike avatar Mar 31 '21 06:03 sysadminmike

I was thinking something similar. I was looking at pg_hba yesterday and not sure what it should be. here is what I have, do you know how I should change it?

IPv4 local connections:

host all all 0.0.0.0/0 trust

IPv6 local connections:

host all all ::1/128 md5

Allow replication connections from localhost, by a user with the

replication privilege.

host replication all 127.0.0.1/32 md5 host replication all ::1/128 md5

jmorganh avatar Mar 31 '21 14:03 jmorganh

I tried but did not work:

#IPv4 local connections: host all all 127.0.0.1/32 trust host all all 0.0.0.0/0 trust

jmorganh avatar Mar 31 '21 16:03 jmorganh

I am not getting the log of "Connected to postgres" from the index.js code. In the part with "pgclient.connect(function(err) {"

on my desktop I get: Connected to postgres couch_import: initial since=7 couch_import: Starting checkpointer couch_import: Checkpoint 7 is current next check in: 120 seconds

running on the server I get: couch_import: Starting checkpointer couch_import: Checkpoint 1 is current next check in: 120 seconds

jmorganh avatar Mar 31 '21 16:03 jmorganh

I think you need to check postgres logs (probably somewhere in /var/log) not the output of daemon.js - postgres should give you some clue as to why its not accepting the connection so is some issue on your sever config rather than anything else.

sysadminmike avatar Mar 31 '21 17:03 sysadminmike

I found some logs in ...data/log but nothing important

Where do I find ../var/log ?

Thanks so much

jmorganh avatar Mar 31 '21 17:03 jmorganh

It depends on your server (ie os type & postgres version) so really dont know where your server has postgres setup to log to - you may get some clues from postgres.conf file.

sysadminmike avatar Mar 31 '21 17:03 sysadminmike

I have been running through a lot of setting with the IP addresses. So, couch-to-postgres(ctp) works great running from a different machine than the postgres DB and I use the ip of the DB in cpt settings. When I run ctp on the same machine as the db what ip should i use in the ctp settings?

Thanks

jmorganh avatar Apr 01 '21 20:04 jmorganh

Also, why do i get all those "follow:http:........" and "follow:stream http:....." messages on the server but not on my desktop ?

Thanks

jmorganh avatar Apr 01 '21 23:04 jmorganh

You should be able to see what ip to set it to from postgres.conf - again i would check postgres logs for any clue as to why you cannot connect from the server running postgres.

If this continues to be a problem and you have other vm's to use I suggest you run ctp on a different machine rather than the machine running postgres as seem to be is a postgres config issue for local connections and not sure what else to suggest.

sysadminmike avatar Apr 02 '21 07:04 sysadminmike

Thanks for all your help. I have tried a lot of different settings in ctp and in pg_hba, nothing working yet.

now what I did figure out is: When i follow "node-postgres", require('pg'), startup on npm (https://node-postgres.com/features/connecting) from the server with the PG, I can connect and run queries. Yaa

When I change the ctp index.js file to match "node-postgres", on the server with PG, it is acting like the query is not running .

So therefore I can connect to PG from the same server but not with ctp.

Any ideas?

jmorganh avatar Apr 02 '21 19:04 jmorganh

I think I have tracked it down a bit: pgclient.connect() = undefined but when I log pgclient I get the entire object including the params I assigned to it.

jmorganh avatar Apr 02 '21 19:04 jmorganh

I got it to work!!! yaaaaaa. It still a little noisy on the console, This is my first time helping make a npm module work better. How do I submit? What i have done is:

  • upgraded npm module pg to "^8.5.1"
  • modified the top of bin/index.js (see below)

And I have your "not connected to PG error" to work also. (line 28 note in /bin/index.js) here is what i have for the top of index.js

const { Pool, Client } = require('pg')
var PostgresCouchDB = require('../lib');

var settings = 
      {
        couchdb: {
         url: 'http://administrator:pass@ipAddress:5984',
         pgtable:  'couch_import',
         database: 'tcsmaster'
       }
      };

var pgclient = new Client({
	user: 'user',
  	host: 'ipAddress',
  	database: 'tcs_import',
  	password: 'pass',
  	port: 5432,
});

pgclient
  .connect()
  .then(() => {
        console.log('Connected to postgres');
	initial_since = get_initial_since(settings.couchdb.pgtable);
	createImporter();
  })
  .catch(err => console.error('connection error', err.stack))

jmorganh avatar Apr 02 '21 20:04 jmorganh