datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

[EPIC] Schema metadata handling / bugs

Open alamb opened this issue 1 year ago • 2 comments

Is your feature request related to a problem or challenge?

As described by @wiedld on https://github.com/apache/datafusion/pull/12729

A https://github.com/apache/datafusion/pull/11989 (and released in 42.0.0) has added a check that the logical and physical plan input schemas (to an aggregate node) are the same.

This check fails for us in several different queries now downstream in InfluxDB and this ticket tracks the list of issues we found

Issues

  • [x] https://github.com/apache/datafusion/issues/12687
  • [x] #12734
  • [x] #12735
  • [ ] https://github.com/apache/datafusion/issues/12736
  • [x] https://github.com/apache/datafusion/pull/12951
  • [ ] https://github.com/apache/datafusion/pull/12954

alamb avatar Oct 03 '24 10:10 alamb

This check fails for us in several different queries now downstream in InfluxDB and this ticket tracks the list of issues we found

If anyone finds other bugs, feel free to add to the epic. 🙏🏼

wiedld avatar Oct 03 '24 17:10 wiedld

@wiedld has found several more issues:

  • [x] https://github.com/apache/datafusion/pull/12951
  • [x] https://github.com/apache/datafusion/pull/12954

alamb avatar Oct 15 '24 22:10 alamb

Found another issue that triggers a schema issue here: https://github.com/apache/datafusion/issues/13190

eejbyfeldt avatar Oct 30 '24 20:10 eejbyfeldt

Found another issue that triggers a schema issue here: #13190

Thanks -- added to the list

Note I now have added the config option to work around the issue in https://github.com/apache/datafusion/pull/13176 for anyone who hits these

alamb avatar Oct 30 '24 21:10 alamb

Found another schema issue here: https://github.com/apache/datafusion/pull/13305. Reproducer test cases are also included.

wiedld avatar Nov 08 '24 08:11 wiedld

The subtasks are largely done , let's call this one done

alamb avatar Aug 12 '25 19:08 alamb

Found another one: https://github.com/apache/datafusion/pull/19335

adriangb avatar Dec 15 '25 16:12 adriangb