firewall413

Added support to add new partitions (/locations) to existing or new Glue tables: Previous PR only registered the Glue table and merely specified its partition columns. What if, for each...

When using external JSON materalization: bumping into default maximum_object_size limit.

1

https://github.com/duckdb/dbt-duckdb/blob/ab16970ba9f616205dcae52a9dcb661c6d8836c6/dbt/include/duckdb/macros/materializations/external.sql#L52 When materializing a table to a JSON file bigger than 30MB, we bump into the following: `Invalid Input Error: "maximum_object_size" of 16777216 bytes exceeded while reading file "s3://xxxxxx.json" (>33554428...

Schema question when performing an external materialization and registering with Glue

3

I'm trying to understand how schema registration works using the Glue.py plugin You run your DBT logic e.g. {{ config(materialized='external',location='s3://mybucket/hello, glue_register=true, ... )}} select 1,2,3 from source After which it...

Fix/register partitioned tables in glue

4

- resolved issue when writing a new partition to glue for a table with the same schema would result in multiple glue schema versions (instead of just adding a partition)...