plugins: add AWS Transcribe and Polly TTS plugin

[dasxran]

This PR makes the following changes:

• Support for AWS Polly Text-to-Speech (TTS)
• Support for AWS Transcribe Speech-to-Text (STT)

[CLAassistant]

All committers have signed the CLA.

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: 32e8f32af6b03ca158f17fec4dda1399b50ba17f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[davidzhao]

thanks for the PR. we'll review this week and hope to get it merged

[dasxran]

Thank you @davidzhao . Moving to Draft to until v0.8.0 changes are done

[TarekWSw]

Any Updates on the AWS Plugin, to merge it ?? I am planning to use livekit but i want to use the AWS plugin

[lusob]

Any Updates on the AWS Plugin, to merge it ?? I am planning to use livekit but i want to use the AWS plugin

+1 I'm using the plugin from months ago without issues

[vanics]

Hye folks, should we consider reviewing this and getting it merged? I am also looking into using AWS Polly

[davidzhao]

Hye folks, should we consider reviewing this and getting it merged? I am also looking into using AWS Polly

will have a look this week. hoping to getting it merged soon

[TarekWSw]

Any Updates on the AWS Plugin, to merge it ?? I am planning to use livekit but i want to use the AWS plugin

+1 I'm using the plugin from months ago without issues

Did you modify anything in the implementation because when i used it, it raised an error

{"message": "unhandled exception while running the job task\nTraceback (most recent call last):\n File "/s2t/transcribe_aws.py", line 40, in entrypoint\n stt_impl = STT()\n ^^^^^\nTypeError: Can't instantiate abstract class STT with abstract method _recognize_impl", "level": "ERROR", "name": "livekit.agents", "pid": 67147, "job_id": "AJ_UZqizXpe4jEn", "timestamp": "2024-12-14T09:01:45.740554+00:00"}

` from livekit.plugins import aws

async def entrypoint(ctx: JobContext): stt_impl = aws.STT()

if not stt_impl.capabilities.streaming:
    stt_impl = stt.StreamAdapter(
        stt=stt_impl,
        vad=silero.VAD.load(
            min_silence_duration=0.2,
        ),
    )

async def transcribe_track(participant: rtc.RemoteParticipant, track: rtc.Track):
    audio_stream = rtc.AudioStream(track)
    stt_forwarder = transcription.STTSegmentsForwarder(
        room=ctx.room, participant=participant, track=track
    )

    stt_stream = stt_impl.stream()
    asyncio.create_task(_forward_transcription(stt_stream, stt_forwarder))

    async for ev in audio_stream:
        stt_stream.push_frame(ev.frame)

`

[davidzhao]

there's been quite a few changes since the PR is originally opened, and I think we'll need to bring it up to date before merging.

@dasxran could you give me write access to this repo? I'd be glad to fix up a few things before merging

[jayeshp19]

@davidzhao I've already created a new branch from this branch with latest changes https://github.com/livekit/agents/tree/aws-tts-stt

[TarekWSw]

@davidzhao I've already created a new branch from this branch with latest changes https://github.com/livekit/agents/tree/aws-tts-stt

Is the version on this branch functional, because it seems to result in the same error !

[jayeshp19]

@TarekWSw Not yet, we will soon merge it to main

[TarekWSw]

@davidzhao @jayeshp19 please let me know if i can help in anything in order to merge this ASAP

[miron-l]

any update on this ?

[meetakshay99]

Any ETA on when this will be available?

[davidzhao]

please follow this PR, which supersedes the current one.