pgloader icon indicating copy to clipboard operation
pgloader copied to clipboard

Max connections reached, increase value of TDS_MAX_CONN , error when triggering data migration with pgloader

Open Prashant-01010110 opened this issue 3 years ago • 0 comments

  • [x] pgloader --version

root@ip-10-0-0-30:~# pgloader --version pgloader version "3.6.2" compiled with SBCL 2.0.6.debian root@ip-10-0-0-30:~#

```
  • [x] did you test a fresh compile from the source tree?

No, but Installed pgloader on debian-11 OS from official debian repositories

  • [x] did you search for other similar issues? Yes, but could not find straight forward answer to the problem

  • [ ] how can I reproduce the bug? I am Trying to Migrate Database from MSSQL to PostgreSQL MSSQL and postgreSQL are on different hosts on Amazon Linux machines pgloader is installed on a another different host with debian-11 os

after this setup, and doing necessary connectivity checks at the network and OS level and ensuring the connectivity works fine, I tested the pgloader tool with following command

following is for the output for reference>>>>>

root@ip-10-0-0-30:~# pgloader --verbose load.file_db 2022-07-29T13:59:59.004000Z NOTICE Starting pgloader, log system is ready. 2022-07-29T13:59:59.010000Z LOG pgloader version "3.6.2" 2022-07-29T13:59:59.119000Z NOTICE Executing SQL block for before load 2022-07-29T13:59:59.136000Z LOG Migrating from #<MSSQL-CONNECTION mssql://[email protected]:1433/movies {1005F92D53}> 2022-07-29T13:59:59.136000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://[email protected]:5432/entertainment {1005F93A93}> Max connections reached, increase value of TDS_MAX_CONN 2022-07-29T13:59:59.538000Z NOTICE Prepare PostgreSQL database. 2022-07-29T13:59:59.608000Z NOTICE Completing PostgreSQL database. 2022-07-29T13:59:59.611000Z NOTICE Reset sequences 2022-07-29T13:59:59.672000Z LOG report summary reset table name errors read imported bytes total time read write


        before load          0          1          1                     0.014s
    fetch meta data          0          0          0                     0.384s
     Create Schemas          0          0          0                     0.003s
   Create SQL Types          0          0          0                     0.007s
      Create tables          0          0          0                     0.002s
     Set Table OIDs          0          0          0                     0.000s


COPY Threads Completion 0 4 4 0.001s Index Build Completion 0 0 0 0.000s Reset Sequences 0 0 0 0.037s Primary Keys 0 0 0 0.000s Create Foreign Keys 0 0 0 0.000s Create Triggers 0 0 0 0.004s Install Comments 0 0 0 0.000s


  Total import time          ✓          0          0                     0.042s

root@ip-10-0-0-30:~# pgloader --version pgloader version "3.6.2" compiled with SBCL 2.0.6.debian

this is the content of load.file_db >>>>>>>>>>

root@ip-10-0-0-30:~# cat load.file_db load database from mssql://pgloader_user:[email protected]:1433/movies into postgresql://pgloader_1:[email protected]:5432/entertainment

including only table names like 'movies' in schema 'dbo'

set work_mem to '16MB', maintenance_work_mem to '512 MB'

before load do $$ drop schema if exists dbo cascade; $$; root@ip-10-0-0-30:~#

I am repeatedly getting this error >>>>> Max connections reached, increase value of TDS_MAX_CONN

the source database MSSQL is very simple, just one table with a few rows and columns and few lines of data filled in.

The requirement is to present a working model for migrating database from MSSQL to postgreSQL with replication and/or continuous migration capabilities this is for working on a POC so we could establish that pgloder can fulfill our data migration needs so we could use it properly with actual data migration.

Support in this regard is highly appreciated, Thanks
[LoadTest.txt](https://github.com/dimitri/pgloader/files/9220769/LoadTest.txt)



Prashant-01010110 avatar Jul 29 '22 14:07 Prashant-01010110

Please see the docs for the FreeTDS driver and its configuration. I believe that the Max connections reached message is mostly harmless.

dimitri avatar Aug 12 '22 21:08 dimitri