astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Unable to render SQL files which have`template_vars` that are lists

Open tatiana opened this issue 2 years ago • 1 comments

Describe the bug

Issue reported by @mag3141592 , while trying to have a template_vars mapping a list.

Version

  • Astro: 0.5.1
  • Database: Snowflake

To Reproduce

  1. Write the DAG containing:
   query_dwh_schema_info = aql.render(
       path='/usr/local/airflow/include/sql/schemas',
       schemas_list=["'SALESFORCE'", "'STRIPE'"])

And the following SQL file, within the declared path:

---
template_vars:
   schemas: schemas_list
---
 
SELECT
   t.table_schema,
   t.table_name,
   CONCAT(t.table_catalog, '.', t.table_schema, '.', t.table_name) table_path
FROM dwh_legacy.information_schema.tables t
WHERE
       t.table_schema IN (schemas)
   AND t.table_type = 'BASE TABLE';
  1. Run the DAG

  2. See error

snowflake.connector.errors.ProgrammingError: 000904 (42000): SQL compilation error: error line 10 at position 27
invalid identifier 'SCHEMAS'
(..
Task exited with return code 1

Expected behavior Render should support multiple types fo template_vars, including string/list/int/etc.

tatiana avatar Mar 12 '22 01:03 tatiana

@dimberman I believe this issue might have been solved with the refactoring done on 0.6.0 - but I haven't tested yet. We could assign this ticket for the next sprint..!

tatiana avatar Mar 12 '22 01:03 tatiana

This not relevant any more. WDYT @tatiana ?

sunank200 avatar Jan 19 '23 09:01 sunank200