dataform icon indicating copy to clipboard operation
dataform copied to clipboard

CLI Output Only shows up to the first "." in the filename

Open BenBirt opened this issue 2 years ago • 1 comments

Original public issue (with attached context): https://b.corp.google.com/issues/242250553

BenBirt avatar Aug 22 '22 15:08 BenBirt

From the linked bug:

Problem you have encountered:

CLI output will not display the fill SQLX filename when a "." is present, instead the output is only showing up to the first "." in the filename.

What you expected to happen:

Display full name up to the file extension ".sqlx"

Steps to reproduce:

Create Operations Definition Files:

"LOAD_CSV.TABLE1.sqlx" "LOAD_CSV.TABLE2.sqlx"

Execute dataform compile

Output would be:

Compiling...

Compiled 2 action(s).
2 operation(s):
  STAGING.LOAD_CSV 
  STAGING.LOAD_CSV

a2wd avatar Feb 13 '23 15:02 a2wd

Playing around with this, I think it almost works as intended. If you allow dots in filenames to be added to action names, it makes the BigQuery table reference invalid.

E.g. STAGING.LOAD_CSV is valid, as it follows dataset.name, but STAGING.LOAD_CSV.TABLE1 would then follow the BigQuery target reference of project.dataset.name, which is entirely different.

So to make it clear why this is invalid, I've filed https://github.com/dataform-co/dataform/pull/1699 to throw an error for it.

Ekrekr avatar Mar 27 '24 11:03 Ekrekr