unable to clone database from source to target using pgcopydb

[sagar-git-hub]

Hi, I pulled the docker image and started the container. I did a export of PGCOPYDB_SOURCE_PGURI and PGCOPYDB_TARGET_PGURI. My target DB is fresh instance.

I first did a pgcopydb clone --table-jobs 16 --index-jobs 16 . I am getting errors( please see attached clone_error_logs.txt clone_error_logs.txt

I then did a pgcopydb clone --resume --table-jobs 16 --index-jobs 16 --not-consistent --no-owner --no-acl

It is trying to create DB objects again and failing with error

08:57:47.438 102 WARN pg_restore: warning: errors ignored on restore: 246 08:57:47.439 102 ERROR Failed to run pg_restore: exit code 1 08:57:47.439 102 ERROR Failed to prepare schema on the target database, see above for details 08:57:47.439 102 ERROR Failed to clone source database, see above for details 08:57:47.518 93 ERROR clone process 102 has terminated Ä6Å

Then I did a pgcopydb clone --resume --table-jobs 16 --index-jobs 16 --not-consistent --no-owner --no-acl --drop-if-exists

it fails with below logs

09:01:34.050 130 ERROR Command was: ALTER TABLE IF EXISTS public.config ALTER COLUMN id DROP DEFAULT; 09:01:34.216 130 ERROR pg_restore: from TOC entry 5008; 2604 18973 DEFAULT address id dbo 09:01:34.216 130 ERROR pg_restore: error: could not execute query: ERROR: relation "public.address" does not exist 09:01:34.216 130 ERROR Command was: ALTER TABLE IF EXISTS public.address ALTER COLUMN id DROP DEFAULT; 09:03:26.581 130 WARN pg_restore: warning: errors ignored on restore: 58 09:03:26.582 130 ERROR Failed to run pg_restore: exit code 1 09:03:26.582 130 ERROR Failed to prepare schema on the target database, see above for details 09:03:26.582 130 ERROR Failed to clone source database, see above for details 09:03:26.642 121 ERROR clone process 130 has terminated Ä6Å

[dimitri]

Please share a full log output from a session. Here there is only a choice of log lines which shows a lot of unrelated errors and the log selection makes it hard to understand where the problems come from.

[sagar-git-hub]

Please find the log file attached output.log

[dimitri]

Thanks. I think it should be obvious given the error messages: to make sense of what's happening we need the full command line that you've been using, including (if any) the filtering setup.

[sagar-git-hub]

postgres verison - PostgreSQL 13.10 on x86_64-pc-linux-gnu, database we use is of AWS hyperscalar

docker pull ghcr.io/dimitri/pgcopydb:latest docker run -it --rm dimitri/pgcopydb:v0.15 bash export PGCOPYDB_SOURCE_PGURI="postgres://user:[email protected]:port/dbname" export PGCOPYDB_TARGET_PGURI="postgres://user:[email protected]:port/dbname"

export PGCOPYDB_LOG_FILENAME="/tmp/output.log"

first did a pgcopydb clone --table-jobs 16 --index-jobs 16 . I am getting errors( please see attached clone_error_logs.txt clone_error_logs.txt

then did a pgcopydb clone --resume --table-jobs 16 --index-jobs 16 --not-consistent --no-owner --no-acl --drop-if-exists

error logs: output.log

[sagar-git-hub]

Hi @dimitri - Please let me know if you need any more information

[danstoner]

@sagar-git-hub Can you attach the full log from the initial clone? The one you attached appears to start when the errors began.

You mentioned "database we use is of AWS hyperscalar" but it appears you are running the databases in Docker. Can you elaborate on the actual database source and targets being used? Run this in the source and target databases:

select version();

Also, if you can, try pgcopydb:v0.16.