astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Document the partial table use case for different decorators

Open sunank200 opened this issue 1 year ago • 1 comments

Describe the bug Currently, the documentation misses the partial table use case for different decorators.

Expected behaviour Add the documentation for the partial table use case for different decorators.

sunank200 avatar Aug 12 '22 12:08 sunank200

What do you mean by partial table? Can you expand on this please?

kaxil avatar Aug 15 '22 11:08 kaxil

@kaxil If you refer to the example dag - example_snowflake_partial_table_with_append.py

transformed_data = transform_data(
        df=extracted_data, output_table=Table(name="homes_data_long")
)

the table is not defined with conn_id, we infer conn_id, database, and schema from first table which is passed to function, and here is the logic of this - https://github.com/astronomer/astro-sdk/blob/a80d95392e8261c14a15ced21c1e3cd0b70ed05a/src/astro/utils/table.py#L73

utkarsharma2 avatar Aug 16 '22 09:08 utkarsharma2

This is done

sunank200 avatar Jan 19 '23 09:01 sunank200