pipecat icon indicating copy to clipboard operation
pipecat copied to clipboard

feat(cartesia): Add word-level timestamp support to STT service

Open jamsea opened this issue 1 month ago • 1 comments

Overview

This PR adds word-level timestamp support to the Cartesia STT service, aligning the implementation with the ElevenLabs STT approach.

Changes

  • ✅ Add include_timestamps parameter to CartesiaLiveOptions (defaults to True)
  • ✅ Store word timestamp data in TranscriptionFrame.result field when available
  • ✅ Add comprehensive documentation for word timestamp structure (word, start, end)
  • ✅ Enhance logging to differentiate transcripts with/without timestamps
  • ✅ Update example (13f-cartesia-transcription.py) to demonstrate accessing word-level timing data
  • ✅ Maintain backward compatibility (result field is optional)

Word Timestamp Format

When include_timestamps=True, the Cartesia API returns:

{
    "text": "transcript text",
    "is_final": true,
    "language": "en",
    "words": [
        {"word": "transcript", "start": 0.0, "end": 0.5},
        {"word": "text", "start": 0.5, "end": 0.8}
    ]
}

Usage Example

live_options = CartesiaLiveOptions(
    model="ink-whisper",
    include_timestamps=True,
)
stt = CartesiaSTTService(api_key=api_key, live_options=live_options)

# Access word timestamps in TranscriptionFrame
if frame.result and "words" in frame.result:
    for word_data in frame.result["words"]:
        print(f'{word_data["word"]} [{word_data["start"]:.3f}s - {word_data["end"]:.3f}s]')

Implementation Notes

  • Follows the same pattern as ElevenLabsRealtimeSTTService
  • Stores entire API response in result field (not just words array)
  • Works for both final and interim transcriptions
  • Conditional logging based on timestamp availability

jamsea avatar Nov 25 '25 12:11 jamsea

Codecov Report

:x: Patch coverage is 0% with 13 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pipecat/services/cartesia/stt.py 0.00% 13 Missing :warning:
Files with missing lines Coverage Δ
src/pipecat/services/cartesia/stt.py 0.00% <0.00%> (ø)
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Nov 25 '25 13:11 codecov[bot]

Closing in favor of: https://github.com/pipecat-ai/pipecat/pull/3192

markbackman avatar Dec 05 '25 15:12 markbackman