target-postgres icon indicating copy to clipboard operation
target-postgres copied to clipboard

Add option to autocreate schema if not present?

Open laurentS opened this issue 3 years ago • 5 comments

If I run target-postgres with postgres_schema set to a schema that does not exist at the start of the job, the code fails with the following error:

target_postgres.exceptions.PostgresError: ('Exception writing records', InvalidSchemaName('schema "tap_csv" does not exist\nLINE 1: CREATE TABLE "tap_csv"."mytable" ();\n                     ^\n'))

I can get around this by manually creating the schema, or possibly adding the schema creation to before_run_sql, but it feels a little cumbersome, especially if I have other SQL to run before execution. Would it make sense to add a new boolean option like autocreate_schema to issue the schema creation command if it is not there? It could be False by default to maintain the current behaviour if needed.

laurentS avatar Feb 20 '21 00:02 laurentS

I had a similar question for the Snowflake target. Andrew suggested it be included in the SQL base code in this project https://github.com/datamill-co/target-snowflake/issues/22

tayloramurphy avatar Mar 23 '21 14:03 tayloramurphy

it feels like schemas should be created by target-postgres. especially if its written explicitly in the config.

LazerJesus avatar Apr 04 '21 19:04 LazerJesus

~Doesn't this target already autocreate schema? (Is this a stale issue? )~

Nevermind! It's the meltano variant that supports schema autocreation. Always confusing :/

andrewcstewart avatar Jul 12 '21 19:07 andrewcstewart

Is this something the community could contribute to? As this is something we're running into as well.

mvgijssel avatar Oct 05 '21 15:10 mvgijssel

Is this issue dead? Would be great to have this feature

r-nyq avatar Oct 05 '22 15:10 r-nyq