pipecat icon indicating copy to clipboard operation
pipecat copied to clipboard

Elevenlabs TTS issue in the latest pipecat version

Open Shoshin23 opened this issue 4 months ago • 0 comments

Hi,

We're using Pipecat with Fly.io. With the latest version (0.0.42), the ElevenLabs TTS seems to be throwing an error constantly, which is stopping the machines from auto-destroying.

I'm putting the error here along with this ticket:

2024-10-08T15:31:45.684 app[3d8d3e2f769568] ams [info] 2024-10-08 15:31:45.683 | ERROR | pipecat.services.elevenlabs:_keepalive_task_handler:372 - ElevenLabsTTSService#0 exception: no close frame received or sent

2024-10-08T15:31:47.080 app[148e2444f23768] ams [info] 2024-10-08 15:31:47.079 | ERROR | pipecat.services.elevenlabs:_keepalive_task_handler:372 - ElevenLabsTTSService#0 exception: no close frame received or sent

2024-10-08T15:31:47.550 app[56830971f5d4d8] ams [info] 2024-10-08 15:31:47.549 | ERROR | pipecat.services.elevenlabs:_keepalive_task_handler:372 - ElevenLabsTTSService#0 exception: received 1012 (service restart); then sent 1012 (service restart)

Could you please help us troubleshoot it?

This is also a sample of my implementation:

    async with aiohttp.ClientSession() as session:
        transport = DailyTransport(
            room_url,
            token,
            "Chatbot",
            DailyParams(
                api_url=daily_api_url,
                api_key=daily_api_key,
                audio_in_enabled=True,
                audio_out_enabled=True,
                camera_out_enabled=True,
                camera_out_width=1024,
                camera_out_height=576,
                vad_enabled=True,
                vad_analyzer=SileroVADAnalyzer(),
                transcription_enabled=True,
            )
        )

        tts = ElevenLabsTTSServiceWithCleanup(
            aiohttp_session=session,
            api_key=os.getenv("ELEVENLABS_API_KEY", ""),
            voice_id=voice,)

        llm = OpenAILLMService(
            api_key=os.getenv("OPENAI_API_KEY"),
            model="gpt-4o")

        sa = SentenceAggregator()
        va = VisionImageFrameAggregator()
        tma_in = LLMUserResponseAggregator(messages)
        tma_out = LLMAssistantResponseAggregator(messages)
        llm_debugger = LLMDebugger()
        transcription_debugger = TranscriptionDebugger()
        ir = UserImageRequester()
        tf = TextFilterProcessor(user_request_answer)
        imgf = ImageFilterProcessor()

        # ura = LLMUserResponseAggregator(messages)
        
        ura = LastQuestionCaptureAggregator(messages, ir)

        pipeline = Pipeline(
            [
                transport.input(),
                ura,
                llm,
                ParallelPipeline([sa, ir, va, llm], [tf, imgf]),
                tts,
                transport.output(),
            ]
        ) 

task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))

We're not doing anything specific with Elevenlabs apart from just calling it or passing it to the pipeline task. Happy to share more information, if needed.

Shoshin23 avatar Oct 08 '24 15:10 Shoshin23