Problem with intent hitting multiple entities

[richardsorensson]

I'm running into a response "Unexpected error during intent recognition" when running wider questions such as "turn of all lights", while a more narrow line as "turn of all office lights" does work.

I've not done any wider troubleshooting yet, but i've started with removing any groups (light / cover) to assure no custom groupings are causing the issues.

Any one running into the same issue and found a solution?

Here's the RAW log details.

Logger: homeassistant.components.assist_pipeline.pipeline Source: components/assist_pipeline/pipeline.py:938 Integration: Assist pipeline (documentation, issues) First occurred: 16:50:11 (2 occurrences) Last logged: 16:53:23

Unexpected error during intent recognition Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 938, in recognize_intent conversation_result = await conversation.async_converse( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/conversation/init.py", line 467, in async_converse result = await agent.async_process( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 157, in async_process response = await self.query(user_input, messages, exposed_entities, 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 280, in query message = await self.execute_function_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 319, in execute_function arguments = json.loads(message["function_call"]["arguments"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 26 column 6 (char 482)

[jekalmin]

You can try increasing response token. This happens because response json is truncated by limit of 150 tokens by default.

Keep in mind that more tokens cost more.

[richardsorensson]

That did the trick, thanks alot!

[richardsorensson]

Did some further testing, with maxed token 4096.

It does not manage to change state of some devices, but while waiting for the chat response "..." after roughly 20-30 seconds, the respons is "Unexpected error during intent recognition", but the entities states are changed first (after just a second or two).

error below

Logger: homeassistant.components.assist_pipeline.pipeline Source: components/assist_pipeline/pipeline.py:938 Integration: Assist pipeline (documentation, issues) First occurred: 22:38:15 (1 occurrences) Last logged: 22:38:15

Unexpected error during intent recognition Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 938, in recognize_intent conversation_result = await conversation.async_converse( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/conversation/init.py", line 467, in async_converse result = await agent.async_process( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 157, in async_process response = await self.query(user_input, messages, exposed_entities, 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 280, in query message = await self.execute_function_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 332, in execute_function return await self.query(user_input, messages, exposed_entities, n_requests) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 280, in query message = await self.execute_function_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 332, in execute_function return await self.query(user_input, messages, exposed_entities, n_requests) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 280, in query message = await self.execute_function_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 332, in execute_function return await self.query(user_input, messages, exposed_entities, n_requests) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 280, in query message = await self.execute_function_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/init.py", line 319, in execute_function arguments = json.loads(message["function_call"]["arguments"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 3 (char 4)

[richardsorensson]

Should mention i ran with ggpt-4-1106-preview and changed to gpt-3.5-turbo-1106.

Now it works without errors again, and proper response.

[jekalmin]

It seems like the same error (token limit) with above, but functions are called multiple times.

• Add logging and see what functions with arguments are called.
• If you want to limit the number of function calls, set Maximum function calls to small number such as 1.
• The reason why many functions are called depends on how you asked and how model would behave. If you want to change how model behaves on a particular question, tell model to do so in prompt.

[jekalmin]

Closing the issue. Feel free to reopen it.