dbt-coves icon indicating copy to clipboard operation
dbt-coves copied to clipboard

Request:`generate sources`: Quote columns that use SQL keywords (such as `GROUP`)

Open jaredx435k2d0 opened this issue 2 years ago • 3 comments

Is your feature request related to a problem? Please describe. Sometimes a column in a table may have the name GROUP, ORDER, START, SCHEMA, TABLE, etc. generate sources doesn't quote these columns, causing issues when dbt is run

Describe the solution you'd like I'd like generate sources to automatically quote column names when they conflict with database keywords

Describe alternatives you've considered Manually modifying the templates with huge if x or x conditions to add quotes if needed

jaredx435k2d0 avatar Feb 06 '23 17:02 jaredx435k2d0

@jaredx435k2d0 we understand your situation, but can't quote all table/column names by default. This can cause trouble with providers such as Snowflake, in which lower or upper casing must be strictly followed when using quotes.

i.e: you have a table named TEST_123 SELECT * FROM test_123 and SELECT * FROM TEST_123 are the same, while when using quotes you must enforce the correct uppercasing of the table name: SELECT * FROM "TEST_123" will work, and SELECT * FROM "test_123" wouldn't

With these details considered, we suggest each user to modify his/her templates at will and, in case you use keywords as table/column names (which we don't recomment), find the "template-way" of covering these cases.

BAntonellini avatar Feb 08 '23 13:02 BAntonellini

Understood. I will note here that I'd anticipate this will be an issue for many users. The Fivetran connector for Salesforce, for example, will create a table called group and order for all users of that connector. This was the impetus for my request. There are also other instances of this.

That said, I understand your position on it and understand that all Fivetran Salesforce users can handle it on their end.

jaredx435k2d0 avatar Feb 08 '23 18:02 jaredx435k2d0

yeah, I understand your point, but IMO it is a bad practice to use reserved words as table names. dont want select * from "select" where "where" = 1 lol

Anyway, this seems like an edge case and it is far better to not quote things for the reason Bruno stated above than to force it.

So, there are a few options for a user:

  1. Manually quote the source table, since you would likely rename the staging table
  2. Change the dbt-coves template so all tables are quoted
  3. Quote the specific table with the problem

Let's see what others in the community think.

noel avatar Feb 08 '23 18:02 noel