pre and post hook execution

[ConstantinoSchillebeeckx]

Hi there! Thanks so much for this project, it looks like it might be a perfect fit for my use-case: I have some Glue catalogs mounted in Redshift as external schemas. Some of the transforms I do on these schemas are compute intensive and are spilling to disk. I'd like to execute them with Athena (and this library!) instead.

I'm running into a few issues ...

executing is comparatively slow, dbt run spends a lot of time executing numerous queries to get a list of views and tables from "awsdatacatalog".INFORMATION_SCHEMA.tables (i.e. it's executing this macro). it's taking a list of all my redshift schemas and executing an athena query on each; all of them return no results because those schemas don't exist in athena

• what is the purpose of this?
• can I disable it?

as part of my dbt project, i'm running an on-end-run hook to set various schema permission, when it executes (it executes numerous sql statements) I get an error like

An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: Only one sql statement is allowed

• is there a way to disable execution of my hook?

[aut0clave]

#60 should dramatically reduce the time taken to query Athena metadata. Not a direct answer to your stated problem, but will mitigate it somewhat.