druid
druid copied to clipboard
INSERT/REPLACE dimension target column types are validated against source input expressions
Description
This PR contains a portion of the changes from the inactive draft PR for integrating the catalog with the Calcite planner https://github.com/apache/druid/pull/13686 from @paul-rogers, tables that are defined in the catalog will have their simple dimension defined column types validated against source input expressions mapped to them during DML INSERT/REPLACE operations. Complex measure types columns will not be validated at this time, this will come in a follow up pr. Also enforcing sealed / non-sealed mode; if a table is sealed, no undefined columns may be added to the table during ingestion. Also addressing remaining comments from https://github.com/apache/druid/pull/15836 and https://github.com/apache/druid/pull/15908. This PR has:
- [x] been self-reviewed.
- [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in licenses.yaml
- [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
- [ ] added integration tests.
- [x] been tested in a test Druid cluster.