intents icon indicating copy to clipboard operation
intents copied to clipboard

Assist not picking up get temperature in area intent

Open blacknell opened this issue 2 years ago • 11 comments
trafficstars

I've asked what is the temperature in the bathroom and it comes back with Temperature is

But asking what is the bathroom temperature works with Bathroom temperature is 17.9 °C

The problematic question pipeline seems to pick up the domain and area correctly but not the entity. I've looked through the sentences and can't be sure which it is using HassGetState or HassClimateGetTemperature

Problematic language processing

intent_output:
  response:
    speech:
      plain:
        speech: Temperature is
        extra_data: null
    card: {}
    language: en
    response_type: query_answer
    data:
      targets: []
      success:
        - name: Bathroom
          type: area
          id: bathroom
      failed: []
  conversation_id: null

Problematic Raw

init_options:
  start_stage: intent
  end_stage: intent
  input:
    text: what is the temperature in the bathroom
  pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
  conversation_id: null
stage: done
run:
  pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
  language: en
  runner_data:
    stt_binary_handler_id: null
    timeout: 30
events:
  - type: run-start
    data:
      pipeline: 01gzmc9kyd6gjsg3e79ne8nekd
      language: en
      runner_data:
        stt_binary_handler_id: null
        timeout: 30
    timestamp: "2023-05-05T08:04:14.368342+00:00"
  - type: intent-start
    data:
      engine: homeassistant
      language: en
      intent_input: what is the temperature in the bathroom
    timestamp: "2023-05-05T08:04:14.368798+00:00"
  - type: intent-end
    data:
      intent_output:
        response:
          speech:
            plain:
              speech: Temperature is
              extra_data: null
          card: {}
          language: en
          response_type: query_answer
          data:
            targets: []
            success:
              - name: Bathroom
                type: area
                id: bathroom
            failed: []
        conversation_id: null
    timestamp: "2023-05-05T08:04:14.405445+00:00"
  - type: run-end
    data: null
    timestamp: "2023-05-05T08:04:14.405673+00:00"
intent:
  engine: homeassistant
  language: en
  intent_input: what is the temperature in the bathroom
  done: true
  intent_output:
    response:
      speech:
        plain:
          speech: Temperature is
          extra_data: null
      card: {}
      language: en
      response_type: query_answer
      data:
        targets: []
        success:
          - name: Bathroom
            type: area
            id: bathroom
        failed: []
    conversation_id: null

Successful language processing

intent_output:
  response:
    speech:
      plain:
        speech: Bathroom temperature is 17.9 °C
        extra_data: null
    card: {}
    language: en
    response_type: query_answer
    data:
      targets: []
      success:
        - name: Bathroom Temperature
          type: entity
          id: sensor.temphumid_bathroom_temperature
      failed: []
  conversation_id: null

blacknell avatar May 05 '23 08:05 blacknell

I am seeing similar:

What is the temperature in abigail's bedroom
Sorry, I couldn't understand that
What is Abigail's bedroom temperature?
Abigail's bedroom temperature is 17.34 °C

However, my raw shows an intent match failure:

stage: done
run:
  pipeline: 01gzhtprzzc4rkgr276g0ykwr4
  language: en
  runner_data:
    stt_binary_handler_id: null
    timeout: 30
events:
  - type: run-start
    data:
      pipeline: 01gzhtprzzc4rkgr276g0ykwr4
      language: en
      runner_data:
        stt_binary_handler_id: null
        timeout: 30
    timestamp: "2023-05-08T23:01:10.691293+00:00"
  - type: intent-start
    data:
      engine: homeassistant
      language: en
      intent_input: What is the temperature in abigail's bedroom
    timestamp: "2023-05-08T23:01:10.691638+00:00"
  - type: intent-end
    data:
      intent_output:
        response:
          speech:
            plain:
              speech: Sorry, I couldn't understand that
              extra_data: null
          card: {}
          language: en
          response_type: error
          data:
            code: no_intent_match
        conversation_id: null
    timestamp: "2023-05-08T23:01:10.797547+00:00"
  - type: run-end
    data: null
    timestamp: "2023-05-08T23:01:10.797721+00:00"
intent:
  engine: homeassistant
  language: en
  intent_input: What is the temperature in abigail's bedroom
  done: true
  intent_output:
    response:
      speech:
        plain:
          speech: Sorry, I couldn't understand that
          extra_data: null
      card: {}
      language: en
      response_type: error
      data:
        code: no_intent_match
    conversation_id: null

OzGav avatar May 08 '23 23:05 OzGav

Anyone else seeing this or understanding something wrong here?

blacknell avatar May 16 '23 10:05 blacknell

I’m assuming the devs will pick this up when they have time

OzGav avatar May 16 '23 23:05 OzGav

I'm getting new (not previously noticed) behaviour.

Both ensuite and workshop are areas and each has a temperature sensor sensor.temphumid_ensuite_temperature & sensor.temphumid_workshop_temperature

what is the temperature in the ensuite returns Temperature is 18.3 °C what is the temperature in the workshop returns Temperature is

Looking at states I noticed the only difference is that the workshop sensor has a friendly name Workshop and the ensuite sensor is Temperature. Changing the friendly name of the workshop sensor to Temperature removes the erroneous behaviour.

This seems to me wrong. The intent logic should be using the device class and area to work out which one.

Make sense?

blacknell avatar May 31 '23 06:05 blacknell

This seems to me wrong. The intent logic should be using the device class and area to work out which one. Make sense?

Yes. Work in progress. In the meantime, what you get is a match for "what is <name> in <area>", which doesn't care about device_class at all.

Also, HassClimateGetTemperature intents are not implemented in Home Assistant yet.

tetele avatar May 31 '23 07:05 tetele

This seems to me wrong. The intent logic should be using the device class and area to work out which one. Make sense?

Yes. Work in progress. In the meantime, what you get is a match for "what is <name> in <area>", which doesn't care about device_class at all.

When you say <name> you're referring to friendly name?

Where are things like work in progress defined/discussed?

blacknell avatar May 31 '23 11:05 blacknell

When you say you're referring to friendly name?

Or aliases. That's how it works.

Where are things like work in progress defined/discussed?

Too many places, perhaps. There are the intents pull requests and discussions sections, there's the discord channel (dev talks only). For device_class + area for sensors in particular, I've personally been working on something I haven't yet pushed, as it's based on a PR that got merged in a dependency, but hasn't been published to PyPi and updated as such in intents.

tetele avatar May 31 '23 13:05 tetele

@blacknell @OzGav can you test if "What is the temp in [area]" works for you ? see the issue I just opened. Thanks.

brewston avatar Dec 28 '23 16:12 brewston

@brewston Unfortunately that does not work for me as only climate entities are supported. I think I recall Paulus saying on a webcast that they were working on how to allow other sensors to feed into this question (all mine are MQTT from either Xiaomi sensors or my Aircon)

OzGav avatar Jan 02 '24 15:01 OzGav

Yes. The challenge is knowing which sensor entity to use. You could have a room sensor and also a computer CPU sensor in same room, for example.

blacknell avatar Jan 02 '24 17:01 blacknell

Yes that’s my understanding of the problem.

OzGav avatar Jan 02 '24 23:01 OzGav