spark
spark copied to clipboard
[SPARK-54405][SQL][Metric View] CREATE command and SELECT query resolution
What changes were proposed in this pull request?
This PR implements the command to create metric views and the analysis rule to resolve a metric view query:
- CREATE Metric view
- Add SQL grammar to support
WITH METRICwhen creating a view - Add dollar-quoted string support for YAML definitions
- Implement CreateMetricViewCommand to analyze the view body
- Use a table property to indicate that the View is a metric view since HIVE has no dedicated table type
- Add SQL grammar to support
- SELECT Metric view
- Update SessionCatalog to parse metric view definitions on read
- Add MetricViewPlanner utility to parse the YAML definition and construct an unresolved plan
- Add ResolveMetricView rule to substitute the dimensions and measures reference to actual expressions
NOTE: This PR depends on https://github.com/apache/spark/pull/53146
Why are the changes needed?
SPIP: Metrics & semantic modeling in Spark
Does this PR introduce any user-facing change?
No
How was this patch tested?
build/sbt "hive/testOnly org.apache.spark.sql.execution.SimpleMetricViewSuite"
build/sbt "hive/testOnly org.apache.spark.sql.hive.execution.HiveMetricViewSuite"
Was this patch authored or co-authored using generative AI tooling?
No
cc @cloud-fan to review
thanks, merging to master!