pgcopydb list table-parts fails when filtering has been used to initialize the catalog

[dcupif]

When filtering has been used for a migration, pgcopydb list table-parts fails with:

# prior to running list table-parts we use filtering
pgcopydb clone --filters filter.ini ...

# at this point the catalog is initialized and list table-parts will fail
pgcopydb list table-parts --schema-name public --table-name pairs --split-tables-larger-than "50GB"

13:50:02.203 13886 INFO   Running pgcopydb version 0.17-1.pgdg22.04+1 from "/usr/bin/pgcopydb"
13:50:02.234 13886 INFO   Using work dir "/tmp/pgcopydb"
13:50:02.235 13886 INFO   Current filtering setup is: {"type":"SOURCE_FILTER_TYPE_NONE"}
13:50:02.235 13886 INFO   Catalog filtering setup is: {"type":"SOURCE_FILTER_TYPE_INCL","include-only-table":[{"schema":"public","name":"pairs"}]}
13:50:02.235 13886 ERROR  Catalogs at "/tmp/pgcopydb/schema/source.db" have been setup for a different filtering than the current command, see above for details
13:50:02.235 13886 ERROR  Failed to initialize pgcopydb internal catalogs

---

pgcopydb version: v0.17

[rimbi]

Hi @dcupif , I am not able to reproduce the case with the 2 commands you mentioned. Is there something missing here?

[ldiqual]

I'm experiencing the same issue while trying to see progress of a clone command:

# Clone command
pgcopydb clone
  --source <source>
  --target <target>
  --no-owner
  --no-acl
  --trace
  --fail-fast
  --drop-if-exists
  --not-consistent
  --dir <dir>
  --filter filters.ini


# filters.ini
[include-only-table]
public.users

# Progress command
pgcopydb list progress
  --source <source>
  --filter filters.ini
  --dir <dir>
  --trace

Output:

09:50:22.587 4307 INFO   Running pgcopydb version 0.17.4.ga53c090 from "/opt/homebrew/Cellar/pgcopydb/HEAD-a53c090/libexec/pgcopydb"
09:50:22.600 4307 DEBUG  copydb.c:360              Change Data Capture data is managed at ".pgcopydb/cdc"
09:50:22.600 4307 INFO   copydb.c:105              Using work dir ".pgcopydb"
09:50:22.600 4307 DEBUG  copydb.c:360              Change Data Capture data is managed at ".pgcopydb/cdc"
09:50:22.600 4307 DEBUG  catalog.c:801             Opening SQLite database ".pgcopydb/schema/source.db" with lib version 3.45.1
09:50:22.600 4307 DEBUG  lock_utils.c:139          Created semaphore 196651 (cleanup with ipcrm -s)
09:50:22.600 4307 TRACE  copydb.c:812              copydb_register_sysv_semaphore[1]: 196651
09:50:22.600 4307 DEBUG  catalog.c:801             Opening SQLite database ".pgcopydb/schema/filter.db" with lib version 3.45.1
09:50:22.600 4307 DEBUG  lock_utils.c:139          Created semaphore 196652 (cleanup with ipcrm -s)
09:50:22.600 4307 TRACE  copydb.c:812              copydb_register_sysv_semaphore[2]: 196652
09:50:22.600 4307 DEBUG  catalog.c:801             Opening SQLite database ".pgcopydb/schema/target.db" with lib version 3.45.1
09:50:22.600 4307 DEBUG  lock_utils.c:139          Created semaphore 196653 (cleanup with ipcrm -s)
09:50:22.600 4307 TRACE  copydb.c:812              copydb_register_sysv_semaphore[3]: 196653
09:50:22.600 4307 SQLite catalog.c:7999            [SQLite] select id, source_pg_uri, target_pg_uri, snapshot,        split_tables_larger_than, split_max_parts, filters,        plugin, slot_name from setup
09:50:22.601 4307 DEBUG  catalog.c:638             Catalog has been setup for source "<source>", target "<target>", snapshot ""
09:50:22.601 4307 INFO   catalog.c:752             Current filtering setup is: {"type":"SOURCE_FILTER_TYPE_NONE"}
09:50:22.601 4307 INFO   catalog.c:753             Catalog filtering setup is: {"type":"SOURCE_FILTER_TYPE_INCL","include-only-table":[{"schema":"public","name":"users"}]}
09:50:22.601 4307 ERROR  catalog.c:757             Catalogs at ".pgcopydb/schema/source.db" have been setup for a different filtering than the current command, see above for details
09:50:22.601 4307 ERROR  cli_list.c:1950           Failed to initialize pgcopydb internal catalogs
09:50:22.601 4307 DEBUG  lock_utils.c:212          ipcrm -s 196653
09:50:22.601 4307 DEBUG  lock_utils.c:212          ipcrm -s 196652
09:50:22.601 4307 DEBUG  lock_utils.c:212          ipcrm -s 196651
09:50:22.601 4307 DEBUG  lock_utils.c:212          ipcrm -s 196650

[jace-ys]

I'm hitting the same issue but with the pgcopydb stream sentinel commands instead. Running clone --follow with --filters causes stream sentinel commands to fail as there's no way to pass --filters to those commands to get the catalog entries to match.