Issues with Siri shortcut for voice assist

[stu247]

iOS device model, version and app version

Model Name: iPad 10th gen, iPhone SE, Apple Watch A2723 Software Version: 16.4.1 (a), 16.4.1 (a), 9.4 App version: 2023.4

Home Assistant Core Version

2023.5.3 Describe the bug

This issue has to do with the Siri shortcut for HA Companion. I’ve tried this on iPad (10th gen, 16.4.1 (a))), iPhone (SE, 16.4.1 (a)), and Apple Watch (A2723, 9.4).

A builtin intent “turn on living room” does not turn on the light but Siri replies “Done”. There is no run in the voice assistant debug page.

A builtin intent “close east bedroom shades” runs correctly and Siri replies “Done”. No debug run.

A custom intent “pause music” runs correctly, but Siri replies with “Sorry, something went wrong. Home Assistant says, “Failed: The operation couldn’t be completed. (ObjectMapper.MapError error 1.)”. No debug run.

All three of these sentences run correctly and have debug runs if I enter the text in the dialog box in the upper right hand corner of HA frontend.

The issues appear to be:

1. Voice commands through Siri shortcut are not showing up in the voice assistants debug assistant as a run. (Even when the command was successful.)
2. Turn on builtin intent is not working for lights.
3. Custom intent works correctly, but HA tells Siri that it did not.

Please let me know if any additional information would be helpful.

To Reproduce See above.

Expected behavior See above.

Screenshots

Additional context

[stu247]

Here is an update to this issue. I am now running HA core version 2023.7.2, iOS version 16.5.1 (c), and companion app version 2023.4 (2023.460). These are the latest versions. All of the issues still exist as described above.

[KwisatzJim]

I'm experiencing a similar issue. I just discovered accidentally that the Assist shortcut will turn lights off, but will not turn them on. Running the Assist shortcut by tapping it and then typing command works fine. But using Assist through Siri on my Macbook Air works fine. I'm running HA 2023.8.1, Companion app 2023.4, iOS 16.5.1. I tried deleting the shortcuts and reinstalling but this has a tendency to screw up the shortcuts if you do it too soon, so I deleted them again and waited a day and reinstalled the shortcuts, still not working. Assist button works fine.

[joaomlourenco]

I have exactly the same problem. Same symptoms. Same behavior. Assist shortcut turns off stuff (lights, sound system) but does not turn them on. Assist button shortcut recognize speech and works all right for both on and off.

[Yakfeet]

I have exactly the same issue, hope it will be fixed soon.

[RyuzakiKK]

I just wanted to point out that this problem is still happening with the latest HA core 2023.12.1 and the companion app 2023.7.

[pissa78]

I have the same issue: HA core 2023.12.3 Companion App 2023.7 (2023.471)

[stu247]

I am now on HA core 2024.1.6 and Companion App 2024.1 (2024.535).

All of the responses from Siri are now positive (OK, Done, or That's done) whether the command executed correctly or not. There is no feedback for a failed command.

Custom intents work correctly. Built-in intents for shades (open and close) also work correctly.

The turn on and off built-in intents still do not work. There is now an error message in the home assistant log:

2024-02-06 14:00:41.032 WARNING (SyncWorker_11) [root] No language set, so cannot convert brightness digits to words

I only have one voice assistant set up and it has the language set to English for all of the assistant options. The voice assistant is working fine from multiple ESP32-S3-Box3 and entering text from the GUI also works, so I don't think there is an issue with language setting.

[stu247]

I tried the Assist Button shortcut. This shortcut works correctly, while the Assist shortcut does not for turn on/off lights and switches. With Assist shortcut, when I try to turn on a light, the light does not turn on and the following three messages appear in the home-assistant.log file:

2024-02-08 09:04:04.128 WARNING (SyncWorker_1) [root] No language set, so cannot convert brightness digits to words
2024-02-08 09:04:04.129 WARNING (SyncWorker_1) [root] No language set, so cannot convert brightness digits to words
2024-02-08 09:04:04.130 WARNING (SyncWorker_1) [root] No language set, so cannot convert brightness digits to words

On the other hand, using Assist Button shortcut the light is turned on and there is no messages in home-assistant.log.

The difference between the two in the shortcuts is that Assist shortcut passes Provided Input to the Assist action while Assist Button shortcut passes Dictated Text.

[collateral87]

I have the same issue with german language and have no idea how to fix this. Very annyoing

[Jack77777777]

Same here, I can not control devices through both shortcuts, no turn on or off. HA Core 2024.2.2 Companion IOS App 2024.1 (2024.535)

[boriken72]

ironically on a whim I changed the last action: Assist with "Provided Input" value to 'Rich Text' and it started to work properly again.

[SkywiperSolutions]

ironically on a whim I changed the last action: Assist with "Provided Input" value to 'Rich Text' and it started to work properly again.

How do you select the variable "Provided Input"? I only have "Ask Each Time".