01
01 copied to clipboard
Published
20 hours ago •
OpenInterpreter
OpenInterpreter
After the first message the voice does not work
Describe the bug The response is not returned to the mobile application(neither text nor voice). Here is copy-paste from the terminal.
> And what can you do?
●
> {START}
I can assist with●
I can assist with a variety of tasks, including web searches, calendar management, email and SMS messaging, file editing, and more. What do you need help with today?●
2024-09-15 15:03:17,633 - ERROR livekit.plugins.elevenlabs - 11labs reported an error: detected_unusual_activity {"pid": 58892, "job_id": "AJ_gWXL8sFCXMze"}
I can assist with a variety of tasks, including web searches, calendar management, email and SMS messaging, file editing, and more. What do you need help with today?
●
2024-09-15 15:03:18,434 - ERROR livekit.plugins.elevenlabs - Error in _run
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 259, in _run
await self._run_ws(word_stream)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 364, in _run_ws
await utils.aio.gracefully_cancel(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 362, in _run_ws
await asyncio.gather(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 341, in recv_task
raise Exception(
Exception: 11labs connection closed unexpectedly, not all tokens have been consumed {"pid": 58892, "job_id": "AJ_gWXL8sFCXMze"}
2024-09-15 15:03:18,438 - ERROR livekit.plugins.elevenlabs - Error in _main_task
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 268, in _main_task
await utils.aio.gracefully_cancel(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 266, in _main_task
await asyncio.gather(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 259, in _run
await self._run_ws(word_stream)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 364, in _run_ws
await utils.aio.gracefully_cancel(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 362, in _run_ws
await asyncio.gather(*tasks)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/plugins/elevenlabs/tts.py", line 341, in recv_task
raise Exception(
Exception: 11labs connection closed unexpectedly, not all tokens have been consumed {"pid": 58892, "job_id": "AJ_gWXL8sFCXMze"}
2024-09-15 15:03:18,439 - ERROR livekit.agents.voice_assistant - Error in _read_generated_audio_task
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 236, in _read_generated_audio_task
handle._tr_fwd.mark_audio_segment_end()
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/transcription/tts_forwarder.py", line 188, in mark_audio_segment_end
self._forming_segments.audio = self._forming_segments.q[0]
I can assist with a variety of tasks, including web searches, calendar management, email and SMS messaging, file editing, and more. What do you need help with today?
Let●
2024-09-15 15:03:18,758 - ERROR livekit.agents.voice_assistant - Error in _stream_synthesis_task
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 251, in _stream_synthesis_task
tts_stream.push_text(seg)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 107, in push_text
self._check_not_closed()
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 130, in _check_not_closed
raise RuntimeError(f"{cls.__module__}.{cls.__name__} is closed")
RuntimeError: livekit.plugins.elevenlabs.tts.SynthesizeStream is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 262, in _stream_synthesis_task
await utils.aio.gracefully_cancel(read_atask)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 236, in _read_generated_audio_task
handle._tr_fwd.mark_audio_segment_end()
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/transcription/tts_forwarder.py", line 188, in mark_audio_segment_end
self._forming_segments.audio = self._forming_segments.q[0]
~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: deque index out of range {"pid": 58892, "job_id": "AJ_gWXL8sFCXMze"}
2024-09-15 15:03:18,760 - ERROR livekit.agents.voice_assistant - Error in _synthesize_task
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 251, in _stream_synthesis_task
tts_stream.push_text(seg)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 107, in push_text
self._check_not_closed()
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 130, in _check_not_closed
raise RuntimeError(f"{cls.__module__}.{cls.__name__} is closed")
RuntimeError: livekit.plugins.elevenlabs.tts.SynthesizeStream is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 170, in _synthesize_task
await utils.aio.gracefully_cancel(synth)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 262, in _stream_synthesis_task
await utils.aio.gracefully_cancel(read_atask)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 236, in _read_generated_audio_task
handle._tr_fwd.mark_audio_segment_end()
I can assist with a variety of tasks, including web searches, calendar management, email and SMS messaging, file editing, and more. What do you need help with today?
Let me●
2024-09-15 15:03:18,762 - ERROR asyncio - Task exception was never retrieved
future: <Task finished name='Task-33' coro=<AgentOutput._synthesize_task() done, defined at /Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py:13> exception=IndexError('deque index out of range')>
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 251, in _stream_synthesis_task
tts_stream.push_text(seg)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 107, in push_text
self._check_not_closed()
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/tts/tts.py", line 130, in _check_not_closed
raise RuntimeError(f"{cls.__module__}.{cls.__name__} is closed")
RuntimeError: livekit.plugins.elevenlabs.tts.SynthesizeStream is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 170, in _synthesize_task
await utils.aio.gracefully_cancel(synth)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 262, in _stream_synthesis_task
await utils.aio.gracefully_cancel(read_atask)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dm/Library/Caches/pypoetry/virtualenvs/01-jZug3wYD-py3.11/lib/python3.11/site-packages/livekit/agents/voice_assistant/agent_output.py", line 236, in _read_generated_audio_task
handle._tr_fwd.mark_audio_segment_end()
I can assist with a variety of tasks, including web searches, calendar management, email and SMS messaging, file editing, and more. What do you need help with today?
Let me know what you'd like to do next.
To Reproduce
- install deps and livekit
- run the server with -
poetry run 01 --server livekit --qr --expose - scan QR code
- start talking
Expected behavior The mobile application responds with voice, there are no errors in the console.
Desktop (please complete the following information):
- OS: macOS Sonoma 14.6.1 (23G93)
- python3.11
