malloy icon indicating copy to clipboard operation
malloy copied to clipboard

Malloy does not recognize postgres array and composite types

Open kylenesbit opened this issue 1 year ago • 4 comments

What happens?

I attempted to read Postgres arrays and composite types as descried in this doc: https://malloydata.github.io/documentation/patterns/reading_nested

However, it seems Malloy does not recognize nested data (arrays and composite types) with postgres. Postgres nested data shows up as a "value" blob in the Malloy schema. And, when I try to reference the nested data, I get a "can not find path to..." error.

To Reproduce

Create a postgres table with an array or composite type. Create a source from the postgres table. View the source in the malloy schema. Reference a composite type field in the source or query.

OS:

Mac OS

Malloy Client:

Local VS Code

Malloy Client Version:

Lastest VS code extension

Database Connection:

Postgres

kylenesbit avatar Oct 25 '23 17:10 kylenesbit

@lloydtabb i'm not sure if this is a bug or a fundamental limitation of the postgres implementation in it's current state.

mtoy-googly-moogly avatar Oct 26 '23 15:10 mtoy-googly-moogly

I think we have some general type issues with Postgres - it seems even types we think we support like 'double precision' are showing up as 'unsupported'

whscullin avatar Oct 26 '23 17:10 whscullin

https://github.com/malloydata/malloy/commit/0cb2399db88dcd84fc084afa089dbef43cfcc44a @skokenes is causing the 'double precision' breakage.

whscullin avatar Oct 26 '23 18:10 whscullin

I have a fix for double precision. My understanding of the issue with "user defined" types in Postgres is that they do not expose a schema, so we are stuck with an opaque data type that we currently treat as a JSON string.

whscullin avatar Oct 27 '23 18:10 whscullin