Add query-parameter support for index/bulk APIs

[javabrett]

trafficstars

Adds processing for configuration properties of the form elasticsearch.index.param.<name>=<value>. The name/value should be a valid query-string parameter name and value for the Index/Bulk APIs, to be applied to API calls.

See:

• https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#docs-index-api-query-params
• https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk-api-query-params

That the original motivation for this PR is to allow configuration of pipeline-processing when writing records, e.g. ?pipeline=mypipelineinelastic. This change itself is not sufficient to allow that, since in the Jest client used by the connector, query-param names are filtered by an enumeration. So to achieve testing of Elasticsearch pipelines, this Jest PR and a snapshot build are required: https://github.com/searchbox-io/Jest/pull/679 .

items worth review:

• Config property name/prefix
• Should separate config be required for Index and Bulk APIs, and maybe even for Delete (tombstone when configured) - if these have different subsets of allowed query-string parameters, or if different behaviour is desired for each operation.

Testing

There's a cp-demo custom branch/build with an updated readme for testing: https://github.com/confluentinc/cp-demo/tree/javabrett/FF-1556-elasticsearch-test . This includes a custom build which incorporates a Jest patch to demonstrate parameter support including ?pipeline for date-based index-name processing.

[ghost]

@confluentinc It looks like @javabrett just signed our Contributor License Agreement. :+1:

Always at your service,

clabot

[cla-assistant[bot]]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}