llm-ui
llm-ui copied to clipboard
quick start guide doesnt stream code blocks
I setup llm-ui following the quick start guide at https://llm-ui.com/docs/quick-start exactly and streaming mostly works, however when it starts to stream a code block, it first waits until the entire code block is finished before displaying it. If its a lot of code, it just looks empty for a few seconds.
Is there some additional setting that I need that is not mentioned in the quickstart guide?
Here is my useLLMOutput hook:
// ...
const { blockMatches } = useLLMOutput({
llmOutput,
fallbackBlock: {
component: MarkdownComponent, // from Step 1
lookBack: markdownLookBack(),
},
blocks: [
{
component: CodeBlock, // from Step 2
findCompleteMatch: findCompleteCodeBlock(),
findPartialMatch: findPartialCodeBlock(),
lookBack: codeBlockLookBack(),
},
],
isStreamFinished: chat.streaming,
});
// ...
and here is my CodeBlock component:
import type { CodeToHtmlOptions } from "@llm-ui/code";
import {
loadHighlighter,
useCodeBlockToHtml,
allLangs,
allLangsAlias,
} from "@llm-ui/code";
// WARNING: Importing bundledThemes increases your bundle size
// see: https://llm-ui.com/docs/blocks/code#bundle-size
// import { bundledThemes } from "shiki/themes";
// import githubDark from "shiki/themes/github-dark.mjs";
import githubLight from "shiki/themes/github-light.mjs";
import { type LLMOutputComponent } from "@llm-ui/react";
import parseHtml from "html-react-parser";
import { getHighlighterCore } from "shiki/core";
import { bundledLanguagesInfo } from "shiki/langs";
import getWasm from "shiki/wasm";
import Grid from "atoms/grid";
const highlighter = loadHighlighter(
getHighlighterCore({
langs: allLangs(bundledLanguagesInfo),
langAlias: allLangsAlias(bundledLanguagesInfo),
themes: [githubLight],
// themes: Object.values(bundledThemes),
loadWasm: getWasm,
})
);
const codeToHtmlOptions: CodeToHtmlOptions = {
theme: "github-light",
lang: "typescript",
// theme: "github-dark",
};
// Customize this component with your own styling
const CodeBlock: LLMOutputComponent = ({ blockMatch }) => {
const { html, code } = useCodeBlockToHtml({
markdownCodeBlock: blockMatch.output,
codeToHtmlOptions,
highlighter,
});
if (html) {
return (
<Grid
overflowY="auto"
py={2}
css={{
"& pre code": {
display: "grid",
fontFamily: "twilio sans mono !important",
fontWeight: 500,
fontSize: ".9rem",
},
}}
>
{parseHtml(html)}
</Grid>
);
} else {
// fallback to <pre> if Shiki is not loaded yet
return (
<Grid overflowY="auto" py={2}>
<pre className="shiki">
<code>{code}</code>
</pre>
</Grid>
);
}
};
export default CodeBlock;
However when I try the demo at https://llm-ui.com/chat, it does stream code blocks. So what setting am I missing?