dbt-duckdb

dbt-duckdb copied to clipboard

I'm trying to make use of the column property in the Source.Table resource type within a custom plugin. Is there a way to access these from a plugin, or will I need to extend the SourceConfig class?

The use case is that I have zipped csv's that do not have column names, and I'd like to load them using the plugin with their associated column names and types.

Below is an example of the source.yml file.

version: 2
sources:
  - name: source_system
    schema: main
    config:
      plugin: zip
      save_mode: ignore
    tables:
      - name: allowance
        identifier: allowance_{{ var('date')}}
        columns:
          - name: Allow_Company
            data_type: text
          - name: Allow_Pointer
            data_type: text

ack, gotcha-- no I don't make that metadata available to the SourceConfig class right now, we will need to add it in; sorry about that!

I can take a shot at extending it and open a PR.

---

From: Josh Wills @.> Sent: Wednesday, August 16, 2023 7:17:22 PM To: jwills/dbt-duckdb @.> Cc: Bewernick, Robert @.>; Author @.> Subject: Re: [jwills/dbt-duckdb] Plugin SourceConfig returning column metadata (Issue #237)

ack, gotcha-- no I don't make that metadata available to the SourceConfig class right now, we will need to add it in; sorry about that! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because

ack, gotcha-- no I don't make that metadata available to the SourceConfig class right now, we will need to add it in; sorry about that!

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/jwills/dbt-duckdb/issues/237*issuecomment-1681426125__;Iw!!LTKUihUYLQ!PSvj0tZ2YZ092xI5R_zHrCpxjvFhhIKlE66tDclOAkEmLiSdIW328wVQT-CokJ_RFe-4N1lDII_KemMFuldfhOqJMXv9Cto06Q$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABXAMGUSKUNHPJQPGCE2IRDXVVPJFANCNFSM6AAAAAA3S7MUEE__;!!LTKUihUYLQ!PSvj0tZ2YZ092xI5R_zHrCpxjvFhhIKlE66tDclOAkEmLiSdIW328wVQT-CokJ_RFe-4N1lDII_KemMFuldfhOqJMXsz5y7NEA$. You are receiving this because you authored the thread.Message ID: @.***>