astro-sdk
astro-sdk copied to clipboard
Refactor all tests for full coverage
trafficstars
As a step towards "maturing" the astro DAG authoring project, we must rewrite our tests to ensure that every integration test runs against every database.
This step will simultaneously reduce the number of tests we need to maintain, make testing much simpler as we add new databases, and will make a future refactor much simpler as we can ensure proper coverage.
To do this, we will take advantage of two features in pytest, fixtures and parameterize.
The result is that each astro function should have tests that look like this:
@pytest.mark.parametrize(
"sql_server",
[
"bigquery",
"snowflake",
"postgres",
"sqlite",
],
indirect=True,
)
@pytest.mark.parametrize(
"merge_parameters",
[
# "None",
"single",
"multi",
"update",
],
indirect=True,
)
def test_merge(sql_server, sample_dag, tmp_table, merge_parameters):
sql_type, _ = sql_server
merge_params, mode = merge_parameters
with sample_dag:
output_table = tmp_table
run_merge(output_table, merge_params, mode, sql_type)
test_utils.run_dag(sample_dag)
Where we can create a grid of tests that tests different scenarios against all databases.