Render sql code with parameters in BaseSQLDecoratedOperator

[feluelle]

Description

What is the current behavior?

Currently, when using operators inheriting from BaseSQLDecoratedOperator we are unable to view the rendered SQL in the UI.

closes: #1003 depends on: #1004

What is the new behavior?

• add sql and parameters to BaseSQLDecoratedOperator template_fields
• add .sql to BaseSQLDecoratedOperator template_ext
• add check for template_fields and template_ext

Does this introduce a breaking change?

No.

Checklist

• [x] Created tests which fail without the change (if possible)
• [x] Extended the README / documentation, if necessary

[feluelle]

1, Adding a deprecation warning in transform_file 2. Name the new task transform_task (or something else) - so it is clear the difference in comparison to transform - and it could be used both to load from a file and a string? 3. Should we have some tests?

Yes, actually this makes more sense.

[codecov[bot]]

Codecov Report

Base: 92.57% // Head: 93.50% // Increases project coverage by +0.93% :tada:

Coverage data is based on head (03a1719) compared to base (49c08b9). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #897      +/-   ##
==========================================
+ Coverage   92.57%   93.50%   +0.93%     
==========================================
  Files          91       91              
  Lines        4810     4837      +27     
  Branches      476      480       +4     
==========================================
+ Hits         4453     4523      +70     
+ Misses        260      225      -35     
+ Partials       97       89       -8     

Impacted Files	Coverage Δ
...thon-sdk/src/astro/sql/operators/base_decorator.py	95.65% <100.00%> (+0.87%)	:arrow_up:
python-sdk/src/astro/sql/operators/load_file.py	95.14% <0.00%> (-1.95%)	:arrow_down:
python-sdk/src/astro/databases/base.py	92.51% <0.00%> (+4.84%)	:arrow_up:
python-sdk/src/astro/files/locations/amazon/s3.py	92.72% <0.00%> (+5.45%)	:arrow_up:
python-sdk/src/astro/utils/table.py	100.00% <0.00%> (+5.71%)	:arrow_up:
python-sdk/src/astro/databases/snowflake.py	92.24% <0.00%> (+8.33%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[kaxil]

Some tests are failing!

[feluelle]

I am gonna separate "Fix transform_file" from "render sql code with parameters". It seems rendering is not that easy to get working for all cases such as cases containing dataframes.

[tatiana]

I've worked on this PR independently and with @feluelle, and the tests are now passing.

I also validated that both aql. transform and aql.transform_file are rendering the SQL query in the UI:

Example of aql.transform_file:

Example of aql.transform:

Since it was long approved, I'm merging this PR as soon as the CI checks pass - before it completes its five months anniversary. We can continuously improve and iterate over it with time.