intents icon indicating copy to clipboard operation
intents copied to clipboard

One light tests producing errors

Open stevoh6 opened this issue 2 years ago • 13 comments

HI, when I was trying to test enablid/disablid one light entity and write test for it, I ended up with following situation, could you please check it?

tests/sk/light_HassTurnOff.yaml

   - sentences:
       - "vypni nočnú lampu"
       - "zahas nočnú lampu"
       - "zhasni nočnú lampu"
     intent:
       name: "HassTurnOff"
       slots:
         name: "light.bedroom_lamp"
         domain: "light"

Produce error: FAILED tests/test_language_sentences.py::test_light_HassTurnOff[sk] - AssertionError: Slot domain was not expected for: vypni nočnú lampu

But if I remove 'domain' it throw another error. tests/sk/light_HassTurnOff.yaml

   - sentences:
       - "vypni nočnú lampu"
       - "zahas nočnú lampu"
       - "zhasni nočnú lampu"
     intent:
       name: "HassTurnOff"
       slots:
         name: "light.bedroom_lamp"
         # domain: "light"

Produce error: FAILED tests/test_language_sentences.py::test_light_HassTurnOff[sk] - AssertionError: File light_HassTurnOff: tests should have domain slot set to light

stevoh6 avatar Jan 26 '23 10:01 stevoh6

The problem here should not be underestimated. I guess you are always affected by this problem when you have a room with more than one light (wild guess?)

Let's take my config as example:

light.kuchenspots (name: Küchenspots) in area "Küche" light.kuchenlamp (name: Küchenlampe) in area "Küche"

When I say "Dimme die Küchenspots auf 20%" (Dim the küchenspots to 20%) [I stay with my native language here to prevent missunderstandings] it dims every light that has "Spot" in it to 20% - all of them.

The problem is, that I don't have another option to toggle these lights individually.

Let's assume that the lights are now called:

light.kuchenspots (name: Spots) in area "Küche" light.kuchenlamp (name: Lamp) in area "Küche"

And I say: "Dimme die Spots in der Küche auf 20%" (Dim the spots in the kitchen to 20%); HomeAssistant does not understand this intend, because this case is not declared.

As already mentioned in #864

mxwi avatar Feb 03 '23 11:02 mxwi

I encountered the same error. In my view it should be possible to define sentences to turn on/off one specific device/entity outside homeassistant_HassTurnOn/Off.

dontinelli avatar Feb 06 '23 11:02 dontinelli

Solution is to add

requires_context: 
  domain: light

As in #946

ping @stevoh6 @dontinelli @mxwi

schizza avatar Feb 07 '23 20:02 schizza

thx :) I will have time in end of the week look at it and other chenges :)

stevoh6 avatar Feb 07 '23 21:02 stevoh6

@schizza: I see that your PR #946 passed all tests. When I try to do the same (with adding a sentence in the test file), I still get an Error:

E                   AssertionError: Slot domain was not expected for: Schaut d Schlafzimmer Lampe y
E                   assert 'domain' in {'name': 'light.bedroom_lamp'}

tests/test_language_sentences.py:129: AssertionError
=========================================================================== short test summary info ============================================================================
FAILED tests/test_language_sentences.py::test_light_HassTurnOn[de-CH] - AssertionError: Slot domain was not expected for: Schaut d Schlafzimmer Lampe y
================================================================== 1 failed, 1 passed, 17 deselected in 0.25s ==================================================================
intents:
  HassTurnOn:
    data:
      - sentences:
          - "(Zünd|Mach|Schalt|Schaut) [<Liecht>] <name> [<a_y>]"
        slots:
          domain: light
        requires_context:
          domain: light
        response: light
tests:
  - sentences:
      - "Zünd d Schlafzimmer Lampe a"
      - "Schaut d Schlafzimmer Lampe y"
    intent:
      name: HassTurnOn
      slots:
        name: light.bedroom_lamp
        domain: light
    response:
      - "Ha ds Liecht schlafzimmer lampe igschalte."

The test passes, if I delete the sentences for turning on one light only (as you know from the last PR).

Furthermore, the test passes if the test sentence is "Zünd ds Liecht Schlafzimmer Lampe a". It therefore seems that the problem is in the similar sentence for the general HassTurnOn, even though lights should be excluded:

intents:
  HassTurnOn:
    data:
      - sentences:
          - "(Schalt|Schaut|Schteu|Stell|Mach) <name> <a_y>"
        excludes_context:
          domain: (light|fan)

What do I miss?

dontinelli avatar Feb 08 '23 13:02 dontinelli

@schizza: I see that your PR #946 passed all tests. When I try to do the same (with adding a sentence in the test file), I still get an Error:

E                   AssertionError: Slot domain was not expected for: Schaut d Schlafzimmer Lampe y
E                   assert 'domain' in {'name': 'light.bedroom_lamp'}

tests/test_language_sentences.py:129: AssertionError
=========================================================================== short test summary info ============================================================================
FAILED tests/test_language_sentences.py::test_light_HassTurnOn[de-CH] - AssertionError: Slot domain was not expected for: Schaut d Schlafzimmer Lampe y
================================================================== 1 failed, 1 passed, 17 deselected in 0.25s ==================================================================
intents:
 HassTurnOn:
   data:
     - sentences:
         - "(Zünd|Mach|Schalt|Schaut) [<Liecht>] <name> [<a_y>]"
       slots:
         domain: light
       requires_context:
         domain: light
       response: light
tests:
 - sentences:
     - "Zünd d Schlafzimmer Lampe a"
     - "Schaut d Schlafzimmer Lampe y"
   intent:
     name: HassTurnOn
     slots:
       name: light.bedroom_lamp
       domain: light
   response:
     - "Ha ds Liecht schlafzimmer lampe igschalte."

The test passes, if I delete the sentences for turning on one light only (as you know from the last PR).

Furthermore, the test passes if the test sentence is "Zünd ds Liecht Schlafzimmer Lampe a". It therefore seems that the problem is in the similar sentence for the general HassTurnOn, even though lights should be excluded:

intents:
  HassTurnOn:
    data:
      - sentences:
          - "(Schalt|Schaut|Schteu|Stell|Mach) <name> <a_y>"
        excludes_context:
          domain: (light|fan)

What do I miss?

Actually, if I run tests on your branch only error is that you have a wrong answer:

      AssertionError: Incorrect response for: Zünd d Schlafzimmer Lampe a
E                   assert 'Ha ds Liächt schlafzimmer lampe igschalte.' in {'Ha ds Liecht schlafzimmer lampe igschalte.'}

schizza avatar Feb 08 '23 16:02 schizza

@dontinelli

so the problem is that sentence "Schaut d Schlafzimmer Lampe y" is falling to homeassistant_HassTurnOn instead to light_HassTurnOn Sentence "Zünd d Schlafzimmer Lampe a" dont fall into homeassistant_HassTurnOn because you dont have such rule in homeassistant_HassTurnOn sentences for Zund, but for Schaut you have. In homeassistatnt_HassTurnOn is exclude_content but in wrong format (fan|light). Change it only to light and tests will pass as the sentence "Zünd d Schlafzimmer Lampe a" will be parsed in right place.

as you can see here:

{
  "text": "Zünd d Schlafzimmer Lampe a",
  "match": true,
  "intent": "HassTurnOn",
  "slots": {
    "name": "light.bedroom_lamp",
    "domain": "light"
  },
  "response_key": "light",
  "response": "Ha ds Liächt schlafzimmer lampe igschalte."
}

and on other hand:

{
  "text": "Schaut d Schlafzimmer Lampe y",
  "match": true,
  "intent": "HassTurnOn",
  "slots": {
    "name": "light.bedroom_lamp"
  },
  "response_key": "default",
  "response": "Has schlafzimmer lampe igschalte."
}

schizza avatar Feb 08 '23 17:02 schizza

Thank you @schizza, it was indeed the wrong format in the exclude_context-syntax.

        excludes_context:
          domain: light, fan

With this I (currently) get all tests passed.

dontinelli avatar Feb 08 '23 18:02 dontinelli

so we have to adjust this in every language, correct?

mxwi avatar Feb 10 '23 11:02 mxwi

@dontinelli That should be:

        excludes_context:
          domain:
            - light
            - fan

synesthesiam avatar Feb 20 '23 18:02 synesthesiam

@dontinelli That should be:

        excludes_context:
          domain:
            - light
            - fan

Thank you, in the meanwhile schizza told me the same and I amended accordingly.

dontinelli avatar Feb 20 '23 21:02 dontinelli

Am I missing something?

Ger: "Dimme die Spots in der Küche auf 20%"

Eng: "Dim the spots in the kitchen to 20%."

These statements are still not working for me.

Why?

mxwi avatar May 03 '23 13:05 mxwi

Am I missing something?

Ger: "Dimme die Spots in der Küche auf 20%"

Eng: "Dim the spots in the kitchen to 20%."

These statements are still not working for me.

Why?

"die Spots" is not (yet) defined in German. "die Lichter" or "die Lampen" should work, though. I assume in English it's the same.

dontinelli avatar May 04 '23 09:05 dontinelli