ibis icon indicating copy to clipboard operation
ibis copied to clipboard

feat(risingwave): support struct field access

Open christophediprima opened this issue 11 months ago • 6 comments

Description of changes

Add a specific visit method for RisingWave visit_StructField.

"f{idx}" is not supported like in Postgres.

Use dot and parenthesis annotation as dots only is not working on the first level.

`dataframe.metadata["user.profile.link"].value.url``

Will be translated to:

`(((("t1"."metadata")."user.profile.link").value).url)``

Issues closed

  • Resolves #11182

christophediprima avatar May 08 '25 14:05 christophediprima

This is somewhat tricky because of the way risingwave encodes struct types.

cpcloud avatar May 19 '25 14:05 cpcloud

Thanks for your reply @cpcloud. I don't really get what is the issue with the test that is failing. Can you briefly explain?

christophediprima avatar May 20 '25 07:05 christophediprima

I think I managed to get a first version of this working (meaning passing all existing tests without changing the tests).

cpcloud avatar May 20 '25 12:05 cpcloud

@cpcloud looks great! Thanks a lot for pushing this forward and getting the tests to pass! I know this PR wasn’t fully complete, so your help in finishing things up is much appreciated.

christophediprima avatar May 21 '25 09:05 christophediprima

There are some complications here :)

  1. RisingWave's DESCRIBE returns a shredded schema, which would be fine if it actually segmented field names for fields in structs like this: STRUCT<"a.b.c" INT>. Currently it does not.
  2. I tried querying information_schema.columns, but that has the same problem: it returns the struct with unquoted fields, and sqlglot is not and will not handle the unquoted fields.

I think the changes need to come from RisingWave, in the form of quoting field names in whatever schema metadata is returned.

cpcloud avatar May 27 '25 13:05 cpcloud

Moving to draft until there's a risingwave release with their changes.

cpcloud avatar Jun 10 '25 12:06 cpcloud

Hi there! Can we reopen that one? :)

christophediprima avatar Aug 13 '25 09:08 christophediprima

Going to do this in a separate, new PR.

cpcloud avatar Aug 18 '25 13:08 cpcloud