Support different versions of dbt in our test matrix

[tatiana]

Context

At the moment, we have:

• Python [3.8 - 3.11] x Airflow [2.3 - 2.9]

And we want to introduce dbt to the test matrix.

This would increase significantly the amount of tests being run, without a clear benefit.

Goal

Have one set of integration & unit tests for:

• Airflow [2.3 - 2.9] x Python 3.8 (oldest supported) + latest dbt
• Airflow [2.3 - 2.9] x Python 3.12 (latest) + latest dbt This would reduce the amount of Python versions we are checking against.

Have integration & unit tests for the following combinations:

• Python 3.12 (? TBD based on #964) + Airflow [2.3 - 2.9] x last 3 or 4 releases of dbt (depending on the results of the survey #963)

Acceptance criteria

• [ ] Introduce dbt to our test matrix

It may be worth having an independent virtual env for dbt, to avoid conflicts similar to #944. Another alternative may be to use constraints, as proposed in #965.

Suppose we decide to have separate virtualenvs for Airflow and db. In that case, we will still have to have at least one integration test with dbt and Airflow in the same venv to validate the behaviour of InvocationMode.DBT_RUNNER: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#invocation-modes.

[tatiana]

Perhaps, instead of running every test against the combination of the matrix, we could use -k and run only tests for specific errors related to some of the older versions.