omi icon indicating copy to clipboard operation
omi copied to clipboard

Add Pocket Device Support

Open yconst opened this issue 2 months ago • 4 comments

Add Pocket Device Support

Overview

Implements complete Pocket device support with MP3 recording download, backend processing, and fixes a bug where Pocket recordings were being incorrectly discarded.

What's New

Frontend:

  • BLE protocol implementation for Pocket devices (PKT01_*)
  • Pocket Recordings UI page with download progress tracking
  • WAL sync integration for MP3 files
  • Automatic device detection and connection

Backend:

  • New /v1/pocket/upload-mp3 endpoint
  • MP3→WAV conversion using ffmpeg
  • VAD segmentation and transcription (fal_whisperx)
  • Added pocket to ConversationSource enum

Bug Fix 🐛

Fixed: Pocket recordings being discarded

  • Pocket recordings are user-initiated, so the LLM discard check is now skipped for ConversationSource.pocket
  • Ensures all Pocket recordings appear as conversations in the app

How It Works

  1. User connects to Pocket device via BLE
  2. App downloads MP3 recordings and creates WAL files
  3. Backend converts MP3→WAV, segments with VAD, and transcribes
  4. Conversations appear in main feed with "Pocket" source tag

Files Changed

21 files changed, 1741 insertions(+), 18 deletions(-)

Key new files:

  • app/lib/pages/pocket/pocket_device_page.dart - Pocket recordings UI
  • app/lib/services/devices/pocket_connection.dart - BLE protocol
  • backend/routers/pocket.py - MP3 processing endpoint

Testing

✅ Fully tested end-to-end with physical Pocket device (PKT01_BLUE)

Test: Connect Pocket device → Download recordings → Verify conversations appear in main feed

Breaking Changes

None - purely additive feature

yconst avatar Nov 05 '25 01:11 yconst

if the raw codec of pocket is mp3, i’d like to support it on the backend side.

currently, we have supported: opus, lc3, aac, pcm fyi.

beastoin avatar Nov 10 '25 10:11 beastoin

@yconst is this ready?

aaravgarg avatar Nov 16 '25 22:11 aaravgarg

@yconst is this ready?

Yeap, it is ready

yconst avatar Nov 17 '25 05:11 yconst

if the raw codec of pocket is mp3, i’d like to support it on the backend side.

currently, we have supported: opus, lc3, aac, pcm fyi.

I added backend support for mp3 codec

yconst avatar Nov 17 '25 05:11 yconst

thank @yconst, i would like to test this PR first. do you know what is the fastest way to get a pocket device? anw:

1/ could we make it support real-time? you know omi's 3rd device integration always starts with real-time capability that makes the devices leverage the power of omi's system.

you can check the doc at https://docs.omi.me/doc/integrations

2/ the code for storage syncs is not integrated seamlessly with the current omi code base. in the app, we should use the same page as other omi devices; in the backend, we should use the same routers. of course, we will add support for how to retrieve the audio (mp3) from pocket devices and support the mp3 codec in the backend if needed, but the UI/routers should remain the same.

refs code: app/lib/pages/conversations/sync_page.dart, backend/routers/sync.py

beastoin avatar Dec 03 '25 10:12 beastoin

Yo, how's it going here? @yconst

aaravgarg avatar Dec 12 '25 00:12 aaravgarg