[Feature] support avro format

[liunaijie]

• added avro format and UT
• kafka support avro format and e2e test

Purpose of this pull request

close #4821 same pr with https://github.com/apache/seatunnel/pull/5064

Check list

• [x] Code changed are covered with tests, or it does not need tests for reason:
• [x] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
• [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• [ ] If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• [ ] Update the release-note.

[liunaijie]

pls re-trigger the CI, no failed job, only one cancelled job

[liugddx]

Run mvn spotless:apply and fix code style error.

[liunaijie]

can we retrigger the CI as we has fix the code style error

[liunaijie]

run successful when disable on flink 1.15 https://github.com/liunaijie/seatunnel/actions/runs/5617980130

[liunaijie]

merge the conflict, pls re-trigger the CI

[liunaijie]

https://github.com/liunaijie/seatunnel/actions/runs/5676004114/job/15382134463 this pr run successful in my repo's CI. But get NPE in this CI. And also can't repet the error in local.

[ic4y]

@liunaijie Great commit! Now we just need to resolve the conflicts.

[ic4y]

Once PRs #5303 and #5222 are merged, they will resolve the e2e testing issue.

[liunaijie]

this pr always failed with PaimonIT, and this pr https://github.com/apache/seatunnel/pull/5419 has fixed this issue, once this issue merged, pls re-trigger the CI, thanks.

[liunaijie]

please help review the pr @liugddx @ic4y @ruanwenjun

the CI run successed. https://github.com/liunaijie/seatunnel/actions/runs/6322524273/job/17501939403

[liunaijie]

@liunaijie good pr, please fix the conflict

@hailin0 Done. PTAL, thanks.

[ruanwenjun]

Basically LGTM, wait ci pass

[liunaijie]

Basically LGTM, wait ci pass

@ruanwenjun PTAL

[zhilinli123]

hi I found some issues in kafka avro e2e testing. My configuration is as follows

source {
  Kafka {
    bootstrap.servers = "kafkaCluster:9092"
    topic = "test_avro_topic"
    result_table_name = "kafka_table"
 // When configured this will pull the consumption from kafka's starting offset kafka.auto.offset.reset = "earliest" does not take effect
    **start_mode = "earliest"** 
    format = avro
    format_error_handle_way = skip
    schema = {
      fields {
        id = bigint
        c_map = "map<string, smallint>"
        c_array = "array<tinyint>"
        c_string = string
        c_boolean = boolean
        c_tinyint = tinyint
        c_smallint = smallint
        c_int = int
        c_bigint = bigint
        c_float = float
        c_double = double
        c_decimal = "decimal(2, 1)"
        c_bytes = bytes
        c_date = date
        c_timestamp = timestamp
      }
    }
  }
}

sink {
  Console {
    source_table_name = "kafka_table"
  }
  Assert {
    source_table_name = "kafka_table"
    rules =
      {
        field_rules = [
          {
            field_name = id
            field_type = long
            field_value = [
              {
                rule_type = NOT_NULL
              },
              {
                rule_type = MIN
                rule_value = 0
              },
              {
                rule_type = MAX
                rule_value = 99
              }
            ]
          }
        ]
      }
  }
}

@liunaijie