spark-sql-flow-plugin
spark-sql-flow-plugin copied to clipboard
Visualize column-level data lineage in Spark SQL
example: insert into table bigdata.test_child_lineage select * from bigdata.test_child_dt where ds='20211207' There is a lineage relationship between test_child_lineage and test_child_dt
Lineage Data can be directly output in JSON format? which is convenient to store in the graph database。
```[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-sql-flow-plugin_2.12: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed: Cannot invoke "java.lang.CharSequence.length()" because "this.text" is null -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute...