Zhen Wang

cc @pan3793 @iodone

> Is it possible to have the lineage plugin applied to the custom spark engine that is not Kyubi Engine? Yes, I would prefer it to remove kyuubi dependencies and...

@iodone , thanks for your review, sorry for replying so late.

> @wForget #3444, In order to implement the `PlanOnly` mode of lineage, imported `SQLParserHelper` with spark-lineage-plugin. This will cause the kyuubi spark engine to strongly depend on spark-lineage-plugin. Are you...

> A general question, why does a spark plugin have to rely on the kyuubi code path? The current lineage plugin is more like a kyuubi spark engine plugin, we...

> Glancing at the comments, I think the author and reviewers reached a consensus the lineage plugin should be a pure Spark plugin and should not depend on `kyuubi-common`, `kyuubi-event`...

seems unrelated failure: ``` AdminResourceSuite: - delete engine - user share level *** FAILED *** List("serviceUri=localhost:32971;version=1.7.0-SNAPSHOT;refId=8bd0f165-37a6-49cf-af67-d82f7a0b3cbb;sequence=0000000000") had size 1 instead of expected size 0 (AdminResourceSuite.scala:103) ```

> > seems unrelated failure: > > ``` > > AdminResourceSuite: > > - delete engine - user share level *** FAILED *** > > List("serviceUri=localhost:32971;version=1.7.0-SNAPSHOT;refId=8bd0f165-37a6-49cf-af67-d82f7a0b3cbb;sequence=0000000000") had size 1 instead...

> In my case, this failure case still exists with re-running It is being followed up in #3753, let's try again later.

This change doesn't seem to be compatible with Spark 3.1, can you make it compatible?