pgloader icon indicating copy to clipboard operation
pgloader copied to clipboard

Clause 'downcase identifiers' seems to be enabled by default.

Open pandrews-hdai opened this issue 2 years ago • 1 comments

I've been trying to migrate an AWS environment from Aurora MySQL to Aurora Serverless PostgreSQL and when checking on the migrated DB it seems that pgloader has implicitly activated 'downcase identifiers' and cast all my columns to lower case even though I didn't have that flag in my WITH clause.

I didn't put 'downcase identifiers' into my file, but all the column names in the postgres DB created by pgloader are lower case. I've had to rename about a dozen manually because it would be more time consuming to lowercase all the queries instead.

Thanks

  • [X] pgloader --version

      pgloader --version
      pgloader version "3.6.2c52da1"
      compiled with Clozure Common Lisp Version 1.12.1 (v1.12.1) LinuxX8664
    
  • [X] did you test a fresh compile from the source tree?

    Yes this occurred using a fresh compile from source.

  • [X] did you search for other similar issues?

  • [X] how can I reproduce the bug?

LOAD DATABASE
  FROM mysql://a_user:[email protected]/mydb?useSSL=true
  INTO postgresql://a_admin:[email protected]/mydb?sslmode=allow

  WITH include drop, create tables, create indexes, reset no sequences, single reader per thread, workers = 4, concurrency = 1, prefetch rows = 100000, batch rows = 100000, on error resume next
  --
  SET MySQL PARAMETERS net_read_timeout = '1800', net_write_timeout = '1800'

EXCLUDING TABLE NAMES MATCHING ~<big_table_that_crashes_pgloader>

ALTER SCHEMA 'mydb' RENAME TO 'public'
;
  • [X] pgloader output you obtain

I only get NOTICE output that copy progress is being made. No other logging. I actually can't get any more output even if I pass --verbose and --debug, the most I get are these NOTICEs.

./pgloader/build/bin/pgloader --verbose perf-migration.load | tee outfile.log

pandrews-hdai avatar Sep 01 '22 21:09 pandrews-hdai

Also with the pre-built pgloader: pgloader version "3.6.2" compiled with SBCL 1.4.0-1.el7

pandrews-hdai avatar Sep 12 '22 16:09 pandrews-hdai

just faced the same issue and there seems to be no way to disable downcase?

RuriRyan avatar Dec 21 '22 13:12 RuriRyan

@RuriRyan I had gotten some mileage out of also setting the "quote identifiers" flag in addition, because not downcasing is apparently not the same configuration as maintaining existing case and the documentation isn't exactly clear.

pandrews-hdai avatar Jan 13 '23 14:01 pandrews-hdai