druid icon indicating copy to clipboard operation
druid copied to clipboard

INSERT/REPLACE dimension target column types are validated against source input expressions

Open zachjsh opened this issue 1 year ago • 0 comments

Description

This PR contains a portion of the changes from the inactive draft PR for integrating the catalog with the Calcite planner https://github.com/apache/druid/pull/13686 from @paul-rogers, tables that are defined in the catalog will have their simple dimension defined column types validated against source input expressions mapped to them during DML INSERT/REPLACE operations. Complex measure types columns will not be validated at this time, this will come in a follow up pr. Also enforcing sealed / non-sealed mode; if a table is sealed, no undefined columns may be added to the table during ingestion. Also addressing remaining comments from https://github.com/apache/druid/pull/15836 and https://github.com/apache/druid/pull/15908. This PR has:

  • [x] been self-reviewed.
    • [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • [ ] added documentation for new or modified features or behaviors.
  • [ ] a release note entry in the PR description.
  • [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • [ ] added or updated version, license, or notice information in licenses.yaml
  • [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • [ ] added integration tests.
  • [x] been tested in a test Druid cluster.

zachjsh avatar Feb 24 '24 02:02 zachjsh