astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Refactor all tests for full coverage

Open dimberman opened this issue 3 years ago • 0 comments
trafficstars

As a step towards "maturing" the astro DAG authoring project, we must rewrite our tests to ensure that every integration test runs against every database.

This step will simultaneously reduce the number of tests we need to maintain, make testing much simpler as we add new databases, and will make a future refactor much simpler as we can ensure proper coverage.

To do this, we will take advantage of two features in pytest, fixtures and parameterize.

The result is that each astro function should have tests that look like this:

@pytest.mark.parametrize(
    "sql_server",
    [
        "bigquery",
        "snowflake",
        "postgres",
        "sqlite",
    ],
    indirect=True,
)
@pytest.mark.parametrize(
    "merge_parameters",
    [
        # "None",
        "single",
        "multi",
        "update",
    ],
    indirect=True,
)
def test_merge(sql_server, sample_dag, tmp_table, merge_parameters):
    sql_type, _ = sql_server
    merge_params, mode = merge_parameters
    with sample_dag:
        output_table = tmp_table
        run_merge(output_table, merge_params, mode, sql_type)
    test_utils.run_dag(sample_dag)

Where we can create a grid of tests that tests different scenarios against all databases.

dimberman avatar Mar 10 '22 19:03 dimberman