Allow user to load context, instead of maintaining context.

[earonesty]

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch @mlc-ai/[email protected] for the project I'm working on. I use the context_free setting and then i stuff the conversation with as many messages as i want, by calling resetChat(), getPipeline(), and appendMessage() as needed. I know it's a hack, but it's a useful one, because compiling web-llm from scratch is challenging.

Making this change "real" would require a new "generate()" function: generateFromHistory(), as well as the pipeline change below... so the user doesn't need to access the private pipeline.

If that's something you're interested in... I will make the change nicely. If not, feel free to use this for anyone that needs it.

Here is the diff that solved my problem:

diff --git a/node_modules/@mlc-ai/web-llm/lib/config.d.ts b/node_modules/@mlc-ai/web-llm/lib/config.d.ts
index cd1a8a3..d26c0a9 100644
--- a/node_modules/@mlc-ai/web-llm/lib/config.d.ts
+++ b/node_modules/@mlc-ai/web-llm/lib/config.d.ts
@@ -24,6 +24,7 @@ export interface ChatConfig {
     repetition_penalty: number;
     top_p: number;
     temperature: number;
+    context_free: boolean;
 }
 export interface ModelRecord {
     model_url: string;
diff --git a/node_modules/@mlc-ai/web-llm/lib/index.js b/node_modules/@mlc-ai/web-llm/lib/index.js
index 4c80ff6..b42ddd4 100644
--- a/node_modules/@mlc-ai/web-llm/lib/index.js
+++ b/node_modules/@mlc-ai/web-llm/lib/index.js
@@ -4207,7 +4207,7 @@ class LLMChatPipeline {
         let tokens = [];
         let prompts;
         // beginning of the conversation
-        if (this.conversation.messages.length <= 2) {
+        if (this.conversation.messages.length <= 2 || this.config.context_free) {
             if (this.conversation.config.add_bos) {
                 tokens = [this.bosTokenId];
             }

This issue body was partially generated by patch-package.