Add native operation for ingest pipeline creation

[DJRickyB]

trafficstars

We should have a native operation in Rally for adding ingest pipelines.

example:

{
  "name": "insert-pipelines",
  "operation-type": "create-pipeline",
  "pipelines": "pipelines/*"
}

Note pipelines above as a file glob, where we'd have API compliant pipeline JSON we could simply save by name to the cluster using the ES Client ingest.put_pipeline operation.

Example working configuration of a custom runner, as a basic example:

import glob
import json
import os

cwd = os.path.dirname(__file__)

async def create_pipeline(es, params):
    pipeline_num = 0
    for pipeline_path in glob.glob(
        os.path.join(cwd, "../", params.get("pipelines", "./pipelines"), "*.json")
    ):
        pipeline_name = os.path.splitext(os.path.basename(pipeline_path))[0]
        with open(pipeline_path, "r") as pipeline_file:
            pipeline = json.load(pipeline_file)
            await es.ingest.put_pipeline(pipeline_name, body=pipeline)
            pipeline_num += 1
    return pipeline_num, "ops"

[namankaushik9876]

Hi, I want to work on this issue. Give me chance.

[pquentin]

We don't generally assign issues, you are welcome to give it a try.