dbt-athena
dbt-athena copied to clipboard
dbt seed fails when too long content
When trying to add a seed file that's too long for Athena, the operation fails because it exceeds the query string length hard limit of 262144 bytes. The error message is somewhat like this:
An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: 1 validation error detected: Value 'insert into database.table (col1, col2, ...) values (........) at 'queryString' failed to satisfy constraint: Member must have length less than or equal to 262144
Perhaps it's not a good use of seeds when the table size exceeds single query lenght limits, but at least FYI for those wondering about the same issue. I worked around this issue by introducing my static table as a new Glue table from S3 data.