[Data stream lifecycle] Conditionally display effective retention

[gmarouli]

While the global retention is work in progress, we would like to display the effective retention conditionally. For this reason we added an extra requirement to determine if we are going to add the field in XContent serialisation.

A sample yaml test for this could be:

---
"Get effective retention from component template":

  - do:
      cluster.put_component_template:
        name: test-component-template
        body:
          template:
            lifecycle:
              data_retention: "10d"
  - is_true: acknowledged

  - do:
      cluster.get_component_template:
        name: test-component-template
  - match: { component_templates.0.component_template.template.lifecycle.effective_retention: "10d" }
  - match: { component_templates.0.component_template.template.lifecycle.retention_determined_by: "data_stream_configuration" }

---
"Get effective retention from index template":

  - do:
      indices.put_index_template:
        name: test-composable-template
        body:
          index_patterns: "my-data-stream"
          template:
            lifecycle:
              data_retention: "10d"
          data_stream: {}
  - is_true: acknowledged

  - do:
      indices.get_index_template:
        name: test-composable-template
  - match: {index_templates.0.index_template.template.lifecycle.effective_retention: "10d"}
  - match: {index_templates.0.index_template.template.lifecycle.retention_determined_by: "data_stream_configuration"}

---
"Get effective retention from data stream":
  - do:
      indices.put_index_template:
        name: ds-template
        body:
          index_patterns: "my-data-stream"
          template:
            lifecycle:
              data_retention: 10d
          data_stream: { }
  - is_true: acknowledged

  - do:
      indices.create_data_stream:
        name: my-data-stream
  - is_true: acknowledged

  - do:
      indices.get_data_lifecycle:
        name: "my-data-stream"

  - match: { data_streams.0.lifecycle.effective_retention: "10d" }
  - match: { data_streams.0.lifecycle.retention_determined_by: "data_stream_configuration" }

  - do:
      indices.get_data_stream:
        name: "my-data-stream"

  - match: { data_streams.0.lifecycle.effective_retention:  "10d" }
  - match: { data_streams.0.lifecycle.retention_determined_by: "data_stream_configuration" }

---
"Get effective retention from simulating index template":

  - do:
      indices.put_index_template:
        name: test
        body:
          index_patterns: te*
          template:
            lifecycle:
              data_retention: "10d"
          data_stream: {}
  - do:
      indices.simulate_index_template:
        name: test

  - match: {template.lifecycle.effective_retention: "10d"}
  - match: {template.lifecycle.retention_determined_by: "data_stream_configuration"}

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)