datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Verify if DISTINCT supports all types, incl binary, complex, etc

Open alamb opened this issue 1 year ago • 4 comments

          Wondering if DISTINCT supports all types, incl binary, complex, etc? if its not we should be mentioning it in the doc, and double check it returns a respective error instead of crash/corruption

Originally posted by @comphead in https://github.com/apache/datafusion/pull/11012#discussion_r1649198256

alamb avatar Jun 21 '24 17:06 alamb

take

LorrensP-2158466 avatar Jun 26 '24 16:06 LorrensP-2158466

Questions:

  • By complex type, you mean like: List, Map, Struct,...?
  • Do we need to verify if the results are correct, or that we can run DISTINCT on all the data types?

LorrensP-2158466 avatar Jun 26 '24 18:06 LorrensP-2158466

Thanks @LorrensP-2158466 I think we need to write tests for possible combinations and if there is a crash replace it with not_impl_err!, if the result is incorrect we can open an issue and reference this issue next to the failing test

comphead avatar Jun 26 '24 21:06 comphead

Ah Okey, I was going in the correct direction, thanks!

LorrensP-2158466 avatar Jun 27 '24 16:06 LorrensP-2158466