doris-spark-connector icon indicating copy to clipboard operation
doris-spark-connector copied to clipboard

Spark Doris Connector Release Note 1.3.0

Open JNSimba opened this issue 1 year ago • 3 comments

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

JNSimba avatar Nov 09 '23 14:11 JNSimba

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released image

cjj2010 avatar Nov 29 '23 01:11 cjj2010

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released image

I am the same

milton4code avatar Dec 01 '23 03:12 milton4code

https://repo.maven.apache.org/maven2/org/apache/doris/spark-doris-connector-3.3_2.12/1.3.0/

lordk911 avatar Dec 04 '23 01:12 lordk911