core icon indicating copy to clipboard operation
core copied to clipboard

Published 20 hours ago • verified publisher icon home-assistant

New template merge_response

Open gjohansson-ST opened this issue 1 year ago • 5 comments
trafficstars

Proposed change

Implements a new template merge_response which is to consolidate responses from service calls. See discussion in https://github.com/home-assistant/architecture/discussions/1045 for context.

def merge_response(value: ServiceResponse) -> list[Any]:
    """Merge action responses into single list.

    Checks that the input is a correct service response:
    {
        "entity_id": {str: dict[str, Any]},
    }
    If response is a single list, it will extend the list with the items
        and add the entity_id and value_key to each dictionary for reference.
    If response is a dictionary or multiple lists,
        it will append the dictionary/lists to the list
        and add the entity_id to each dictionary for reference.
    """

A single dict item per entity response

Action call response:

{
    "calendar.sports": {
        "events": [
            {
                "start": "2024-02-27T17:00:00-06:00",
                "end": "2024-02-27T18:00:00-06:00",
                "summary": "Basketball vs. Rockets",
                "description": "",
            }
        ]
    },
    "calendar.local_furry_events": {"events": []},
    "calendar.yap_house_schedules": {
        "events": [
            {
                "start": "2024-02-26T08:00:00-06:00",
                "end": "2024-02-26T09:00:00-06:00",
                "summary": "Dr. Appt",
                "description": "",
            },
            {
                "start": "2024-02-28T20:00:00-06:00",
                "end": "2024-02-28T21:00:00-06:00",
                "summary": "Bake a cake",
                "description": "something good",
            },
        ]
    },
}

Template: {{ merge_response(response_variable) }}

Result:

[
  {
    "description": "",
    "end": "2024-02-27T18:00:00-06:00",
    "entity_id": "calendar.sports",
    "start": "2024-02-27T17:00:00-06:00",
    "summary": "Basketball vs. Rockets",
    "value_key": "events"
  },
  {
    "description": "",
    "end": "2024-02-26T09:00:00-06:00",
    "entity_id": "calendar.yap_house_schedules",
    "start": "2024-02-26T08:00:00-06:00",
    "summary": "Dr. Appt",
    "value_key": "events"
  },
  {
    "description": "something good",
    "end": "2024-02-28T21:00:00-06:00",
    "entity_id": "calendar.yap_house_schedules",
    "start": "2024-02-28T20:00:00-06:00",
    "summary": "Bake a cake",
    "value_key": "events"
  }
]

A single non dict item per entity response

Action call response:

  {
    "binary_sensor.workday": {
      "workday": true,
    },
    "binary_sensor.workday2": {
      "workday": false,
    },
  }

Template: {{ merge_response(response_variable) }}

Result:

  [
    {
      "entity_id": "binary_sensor.workday",
      "workday": true,
    },
    {
      "entity_id": "binary_sensor.workday2",
      "workday": false,
    },
  ]

Multiple items per entity response

Action call response:

  {
    "vacuum.deebot_n8_plus_1": {
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "ok",
        },
      },
    },
    "vacuum.deebot_n8_plus_2": {
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "ok",
        },
      },
    },
  }

Template: {{ merge_response(response_variable) }}

Result:

  [
    {
      "entity_id": "vacuum.deebot_n8_plus_1",
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "ok",
        },
      },
    },
    {
      "entity_id": "vacuum.deebot_n8_plus_2",
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "ok",
        },
      },
    },
  ]

Multiple items per entity response and optionally selecting a single key

Action call response:

  {
    "vacuum.deebot_n8_plus_1": {
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "ok",
        },
      },
    },
    "vacuum.deebot_n8_plus_2": {
      "header": {
        "ver": "0.0.1",
      },
      "payloadType": "j",
      "resp": {
        "body": {
          "msg": "not_ok",
        },
      },
    },
  }

Template: {{ merge_response(response_variable), selected_key='resp' }}

Result:

  [
    {
      "body": {
        "msg": "ok",
      },
      "entity_id": "vacuum.deebot_n8_plus_1",
    },
    {
      "body": {
        "msg": "not_ok",
      },
      "entity_id": "vacuum.deebot_n8_plus_2",
    },
  ]

Optional sorting

Action response:

{
    "weather.forecast_home": {
      "forecast": [
        {
          "condition": "cloudy",
          "datetime": "2024-03-31T10:00:00+00:00",
          "humidity": 71,
          "precipitation": 0,
          "precipitation_probability": 6.6,
          "temperature": 10.9,
          "templow": 6.5,
          "wind_bearing": 71.8,
          "wind_gust_speed": 24.1,
          "wind_speed": 13.7,
        },
        {
          "condition": "cloudy",
          "datetime": "2024-04-01T10:00:00+00:00",
          "humidity": 79,
          "precipitation": 0,
          "precipitation_probability": 8,
          "temperature": 10.2,
          "templow": 3.4,
          "wind_bearing": 350.6,
          "wind_gust_speed": 38.2,
          "wind_speed": 21.6,
        },
        {
          "condition": "snowy",
          "datetime": "2024-04-02T10:00:00+00:00",
          "humidity": 77,
          "precipitation": 2.3,
          "precipitation_probability": 67.4,
          "temperature": 3,
          "templow": 0,
          "wind_bearing": 24.5,
          "wind_gust_speed": 64.8,
          "wind_speed": 37.4,
        },
      ],
    },
    "weather.smhi_home": {
      "forecast": [
        {
          "cloud_coverage": 100,
          "condition": "cloudy",
          "datetime": "2024-03-31T16:00:00",
          "humidity": 87,
          "precipitation": 0.2,
          "pressure": 998,
          "temperature": 10,
          "templow": 4,
          "wind_bearing": 79,
          "wind_gust_speed": 21.6,
          "wind_speed": 11.88,
        },
        {
          "cloud_coverage": 100,
          "condition": "rainy",
          "datetime": "2024-04-01T12:00:00",
          "humidity": 88,
          "precipitation": 2.2,
          "pressure": 999,
          "temperature": 6,
          "templow": 1,
          "wind_bearing": 17,
          "wind_gust_speed": 20.52,
          "wind_speed": 8.64,
        },
        {
          "cloud_coverage": 100,
          "condition": "cloudy",
          "datetime": "2024-04-02T12:00:00",
          "humidity": 71,
          "precipitation": 1.3,
          "pressure": 1003,
          "temperature": 0,
          "templow": -3,
          "wind_bearing": 17,
          "wind_gust_speed": 57.24,
          "wind_speed": 30.6,
        },
      ],
    },
  }

Template: {{ merge_response(response_variable), sort_by='datetime' }}

Result:

  [
    {
      "condition": "cloudy",
      "datetime": "2024-03-31T10:00:00+00:00",
      "entity_id": "weather.forecast_home",
      "humidity": 71,
      "precipitation": 0,
      "precipitation_probability": 6.6,
      "temperature": 10.9,
      "templow": 6.5,
      "value_key": "forecast",
      "wind_bearing": 71.8,
      "wind_gust_speed": 24.1,
      "wind_speed": 13.7,
    },
    {
      "cloud_coverage": 100,
      "condition": "cloudy",
      "datetime": "2024-03-31T16:00:00",
      "entity_id": "weather.smhi_home",
      "humidity": 87,
      "precipitation": 0.2,
      "pressure": 998,
      "temperature": 10,
      "templow": 4,
      "value_key": "forecast",
      "wind_bearing": 79,
      "wind_gust_speed": 21.6,
      "wind_speed": 11.88,
    },
    {
      "condition": "cloudy",
      "datetime": "2024-04-01T10:00:00+00:00",
      "entity_id": "weather.forecast_home",
      "humidity": 79,
      "precipitation": 0,
      "precipitation_probability": 8,
      "temperature": 10.2,
      "templow": 3.4,
      "value_key": "forecast",
      "wind_bearing": 350.6,
      "wind_gust_speed": 38.2,
      "wind_speed": 21.6,
    },
    {
      "cloud_coverage": 100,
      "condition": "rainy",
      "datetime": "2024-04-01T12:00:00",
      "entity_id": "weather.smhi_home",
      "humidity": 88,
      "precipitation": 2.2,
      "pressure": 999,
      "temperature": 6,
      "templow": 1,
      "value_key": "forecast",
      "wind_bearing": 17,
      "wind_gust_speed": 20.52,
      "wind_speed": 8.64,
    },
    {
      "condition": "snowy",
      "datetime": "2024-04-02T10:00:00+00:00",
      "entity_id": "weather.forecast_home",
      "humidity": 77,
      "precipitation": 2.3,
      "precipitation_probability": 67.4,
      "temperature": 3,
      "templow": 0,
      "value_key": "forecast",
      "wind_bearing": 24.5,
      "wind_gust_speed": 64.8,
      "wind_speed": 37.4,
    },
    {
      "cloud_coverage": 100,
      "condition": "cloudy",
      "datetime": "2024-04-02T12:00:00",
      "entity_id": "weather.smhi_home",
      "humidity": 71,
      "precipitation": 1.3,
      "pressure": 1003,
      "temperature": 0,
      "templow": -3,
      "value_key": "forecast",
      "wind_bearing": 17,
      "wind_gust_speed": 57.24,
      "wind_speed": 30.6,
    }
  ]

Type of change

  • [ ] Dependency upgrade
  • [ ] Bugfix (non-breaking change which fixes an issue)
  • [ ] New integration (thank you!)
  • [x] New feature (which adds functionality to an existing integration)
  • [ ] Deprecation (breaking change to happen in the future)
  • [ ] Breaking change (fix/feature causing existing functionality to break)
  • [ ] Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request: https://github.com/home-assistant/home-assistant.io/pull/34050

Checklist

  • [ ] The code change is tested and works locally.
  • [x] Local tests pass. Your PR cannot be merged unless tests pass
  • [x] There is no commented out code in this PR.
  • [x] I have followed the development checklist
  • [x] I have followed the perfect PR recommendations
  • [x] The code has been formatted using Ruff (ruff format homeassistant tests)
  • [x] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • [ ] The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • [ ] New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • [ ] Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

gjohansson-ST avatar Mar 25 '24 21:03 gjohansson-ST

This sounds very similar to the combine filter from ansible. Could help with name inspiration. https://docs.ansible.com/ansible/latest/collections/ansible/builtin/combine_filter.html

Daniel-dev22 avatar Mar 26 '24 00:03 Daniel-dev22

A few comments:

  • The proposed function works for the output from calendar.get_events, but what about other services?
  • Since the output is a list, it should probably be possible for the user to specify how to sort the output
  • We may need an option to specify the expected key of the lists to append
  • We may want to add the source entity_id to each item to make sense of the output

emontnemery avatar Mar 26 '24 07:03 emontnemery

  • The proposed function works for the output from calendar.get_events, but what about other services?

Added some more examples in the tests (but there is limited examples) This should now deal with more than just lists (any type valid for service response should be fine)

  • Since the output is a list, it should probably be possible for the user to specify how to sort the output

Added sorting, will expand it to also deal with something else than dict keys

  • We may need an option to specify the expected key of the lists to append

I understood this as extracting only a single key from a resulting dict so it has been added as single_key

  • We may want to add the source entity_id to each item to make sense of the output

I don't see the use of that?

I wonder a bit about the use of sorting and single_key as both of these are available themselves in templates so are we not just simply making this far more complex than what's normal for templates (keep it simple)?

gjohansson-ST avatar Mar 31 '24 17:03 gjohansson-ST

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days. If you are the author of this PR, please leave a comment if you want to keep it open. Also, please rebase your PR onto the latest dev branch to ensure that it's up to date with the latest changes. Thank you for your contribution!

github-actions[bot] avatar Jun 01 '24 18:06 github-actions[bot]

Docs updated to reflect the changes https://deploy-preview-34050--home-assistant-docs.netlify.app/docs/configuration/templating/#merge-action-responses

gjohansson-ST avatar Aug 20 '24 12:08 gjohansson-ST

As discussed the use of sorting and using selected key has been removed

gjohansson-ST avatar Aug 28 '24 14:08 gjohansson-ST

Test failures are not related to this PR. Merging

gjohansson-ST avatar Sep 02 '24 06:09 gjohansson-ST