doris-spark-connector icon indicating copy to clipboard operation
doris-spark-connector copied to clipboard

[Feature] 优化 spark-doris-connector jar 大小

Open melin opened this issue 1 year ago • 3 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Description

建议优化一下spark-doris-connector jar 大小,当前jar 中包含了很多spark jars目录已经存在jar。 还包含calcite jar 是否必要? image

Use case

No response

Related issues

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

melin avatar Jan 07 '25 01:01 melin

Calcite is referenced by arrow jdbc. Since arrow jdbc is not used yet, the related dependency will be removed in the next version.

gnehil avatar Jan 09 '25 07:01 gnehil

Can eclipse collection be removed?

Calcite is referenced by arrow jdbc. Since arrow jdbc is not used yet, the related dependency will be removed in the next version.

melin avatar Jan 09 '25 08:01 melin

Calcite is referenced by arrow jdbc. Since arrow jdbc is not used yet, the related dependency will be removed in the next version.

commons jar ,spark 中也存在,麻烦去掉,容易导致冲突。

Image Image

melin avatar Feb 12 '25 08:02 melin