Progressive JSON-Based Markdown Rendering with BAML Integration

[codedevote]

I'm trying to prevent direct markdown streaming from the LLM to avoid formatting issues (e.g., extra spaces breaking links). Instead, I'm forcing the LLM to output structured JSON using BAML (https://github.com/BoundaryML/baml). My approach involves defining a schema (see below) where each streamed chunk is a self-contained, possibly incomplete, JSON object that progressively completes. My backend is openai compatible and follows the chunk protocol defined by openai. I am using it with an openai compatible provider in the chatbot.

Schema Example:

baml Copy class Response { sections Section[] }

class Header { level int title string }

class Section { header Header @description(#" This is the header of the section. It is mandatory except there is only a single section. "#) body (TextContent | ImageContent | Section)[] @description(#" No need to enumerate the items, this will be done on the client side "#) }

class TextContent { text string sourceReference SourceReference }

class ImageContent { imageUrl string caption string sourceReference SourceReference }

class SourceReference { url string headerHierarchy Header[] @description(#" This is a list of headers that you referenced in the source document. It is mandatory. "#) pages int[] } Problem: I'm having difficulty determining where to "hack" into the codebase to impact the rendering of standard chat text (non-artifact view). I’ve reviewed the Markdown component, DataStreamHandler, and lib/utils.js, but I'm unclear on which entry point to override or extend to integrate this progressive JSON output.

Request: Could someone advise on:

The optimal hook or integration point in the rendering pipeline for handling these self-contained JSON chunks. Any potential pitfalls when replacing standard markdown rendering with this structured approach. Thanks!