Vector‑Searchable Conversational Memory

[unforced]

✨ Feature Proposal — Vector‑Searchable Conversational Memory

Embed salient chat turns & surface them through the existing POST /search endpoint.

---

1 · Why This Matters

• POST /chat saves entire transcripts in chatThreads.messages (JSONB) but does not embed individual turns.
• POST /search only returns document chunks, so past chat decisions are invisible.
• External tools (Open WebUI, Cursor, Goose, Flowise) want a single memory store that serves both docs and chat snippets.

---

2 · Design Goals

1. Searchable – useful chat turns appear in /search results.
2. Lean & Scalable – skip filler messages; allow configurable policies.
3. Opt‑in & Non‑breaking – defaults keep current behaviour.
4. No new endpoints – reuse POST /search.

---

3 · Proposed Change (high level)

1. Schema – add a child table chat_message_embeddings
   • id bigserial primary key
   • threadId → chatThreads.uuid
   • role (user / assistant), text, ts
   • embedding pgvector(768)
   • metadata JSONB (e.g., { "source": "webui" })
2. Write path (POST /chat)
   • Feature‑flagged by ENABLE_CHAT_MEMORY.
   • After each user turn and assistant reply, call the existing embedding model if shouldEmbed(text, policy) returns true, then insert into chat_message_embeddings.
3. Search path (POST /search)
   • Extend current SQL helper:
     • When body includes "includeChats": true and feature flag on, UNION chat‑message rows with document chunks.
     • Return objects with "src": "doc" | "chat" so clients can badge origin.
4. Config Flags
   • ENABLE_CHAT_MEMORY=false (default)
   • CHAT_EMBED_POLICY=salient (off, all, salient, summary_only)
   • CHAT_EMBED_MIN_LEN=10
   • CHAT_SUMMARY_EVERY=20 (placeholder for future summary tier)

---

4 · Benefits

• Users – Can ask “What did we rename tick() to?” and get a chat citation.
• Tools – Just add "includeChats": true to /search; no extra endpoint.
• Maintainers – Mirrors existing documents → chunks pattern; default opt‑out means zero impact on current installs.

---

5 · Open Questions

• Table name: keep chat_message_embeddings or prefer chat_messages with embedded vector column?
• Default embed policy: salient vs. summary_only.
• Would a richer filter DSL ("type": ["doc","chat"]) be better than the boolean includeChats flag?

---

6 · Next Steps (if accepted)

• Drizzle migration for new table.
• Patch /chat write path under feature flag.
• Refactor search helper and extend POST /search.
• Add README docs & cURL examples.
• (Optional) unit tests for shouldEmbed().

Happy to raise a PR implementing the above once we converge on details.