spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-54405][SQL][Metric View] CREATE command and SELECT query resolution

Open linhongliu-db opened this issue 1 month ago • 1 comments

What changes were proposed in this pull request?

This PR implements the command to create metric views and the analysis rule to resolve a metric view query:

  • CREATE Metric view
    • Add SQL grammar to support WITH METRIC when creating a view
    • Add dollar-quoted string support for YAML definitions
    • Implement CreateMetricViewCommand to analyze the view body
    • Use a table property to indicate that the View is a metric view since HIVE has no dedicated table type
  • SELECT Metric view
    • Update SessionCatalog to parse metric view definitions on read
    • Add MetricViewPlanner utility to parse the YAML definition and construct an unresolved plan
    • Add ResolveMetricView rule to substitute the dimensions and measures reference to actual expressions

NOTE: This PR depends on https://github.com/apache/spark/pull/53146

Why are the changes needed?

SPIP: Metrics & semantic modeling in Spark

Does this PR introduce any user-facing change?

No

How was this patch tested?

build/sbt "hive/testOnly  org.apache.spark.sql.execution.SimpleMetricViewSuite"
build/sbt "hive/testOnly  org.apache.spark.sql.hive.execution.HiveMetricViewSuite"

Was this patch authored or co-authored using generative AI tooling?

No

linhongliu-db avatar Nov 21 '25 20:11 linhongliu-db

cc @cloud-fan to review

linhongliu-db avatar Nov 24 '25 19:11 linhongliu-db

thanks, merging to master!

cloud-fan avatar Dec 17 '25 04:12 cloud-fan