[BitSail][Connector] Support ElasticSearch Source connector.

[BlockLiu]

Is your feature request related to a problem? Please describe

Support ElasticSearch reader.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[liuxiaocs7]

I want to try this, please assign to me, thx.

[BlockLiu]

Nice, please take your time :D

[liuxiaocs7]

In PR #336

I use scroll api to implement paging query.

Each index is now considered a split, may be we coule use the slice parameter to break it down later, just like this link

The job conf looks like this:

{
  "job": {
    "reader": {
      "class": "com.bytedance.bitsail.connector.elasticsearch.source.ElasticsearchSource",
      "es_hosts": ["http://localhost:1234"],
      "es_index": "test1, test2, test3",
      "scroll_size": 3,
      "scroll_time": "1m",
      "columns": [
        {
          "index": 0,
          "name": "id",
          "type": "integer"
        },
        {
          "index": 1,
          "name": "text_type",
          "type": "text"
        },
        {
          "index": 2,
          "name": "keyword_type",
          "type": "keyword"
        },
        {
          "index": 3,
          "name": "long_type",
          "type": "long"
        },
        {
          "index": 4,
          "name": "date_type",
          "type": "date"
        }
      ]
    }
  }
}

@BlockLiu do you think it's ok, thx.

[BlockLiu]

In PR #336

I use scroll api to implement paging query.

Each index is now considered a split, may be we coule use the slice parameter to break it down later, just like this link

The job conf looks like this:

{
  "job": {
    "reader": {
      "class": "com.bytedance.bitsail.connector.elasticsearch.source.ElasticsearchSource",
      "es_hosts": ["http://localhost:1234"],
      "es_index": "test1, test2, test3",
      "scroll_size": 3,
      "scroll_time": "1m",
      "columns": [
        {
          "index": 0,
          "name": "id",
          "type": "integer"
        },
        {
          "index": 1,
          "name": "text_type",
          "type": "text"
        },
        {
          "index": 2,
          "name": "keyword_type",
          "type": "keyword"
        },
        {
          "index": 3,
          "name": "long_type",
          "type": "long"
        },
        {
          "index": 4,
          "name": "date_type",
          "type": "date"
        }
      ]
    }
  }
}

@BlockLiu do you think it's ok, thx.

I think it's a good idea. And from the note below, I think we can get the shard count at first and then build slices.

[BlockLiu]

Namely, we can use shard count as slice number.

[liuxiaocs7]

Namely, we can use shard count as slice number.

Thank you for your suggestion, I will continue to complete