Dbt cannot be run when using a MWAA Connection

[laurodelacerda]

I cannot run my dbt project using a Snowflake Connection.

It is okay with the connection, I tested it using astronomer-cosmos

When I run the using this code as an example it returns that I need to run dbt deps.

So I created a step to run a DbtDepsOperator, and it is all good. But the DbtRunOperator still returns errors.

That is the code.

Errors when run DbtDepsOperator

dbt.exceptions.UninstalledPackagesFoundError: Compilation Error
  dbt found 3 package(s) specified in packages.yml, but only 0 package(s) installed in dbt_packages. Run "dbt deps" to install package dependencies.

dbt.exceptions.DbtRuntimeError: Runtime Error
  fatal: Invalid --project-dir flag. Not a dbt project. Missing dbt_project.yml file

Any help will be much appreciated. 🙏🏼

[millin]

Hi @laurodelacerda Each operator is executed in an isolated environment (temporary directory) and DbtRunOperator knows nothing about the dependencies you have installed. You need to store the project in an external location (e.g. s3) and do an upload it back after the dependencies are installed, because MWAA runs on a readonly filesystem.