Self-deployment, `Text to diagram` AI requests CORS

[rubickecho]

Description

After deploying the fork project to Vercel, configure the Domain as https://excalidraw.xx.com, and then when using the AI "Text to diagram" function, CORS error will occur.

I checked the .env file and the implementation of API requests. I found that replacing only VITE_APP_AI_BACKEND cannot solve the problem. It is also necessary to implement /v1/ai/text-to-diagram/generate API.

VITE_APP_AI_BACKEND=https://oss-ai.excalidraw.com

const response = await fetch(
    `${
        import.meta.env.VITE_APP_AI_BACKEND
    }/v1/ai/text-to-diagram/generate`,
    {
        method: "POST",
        headers: {
        Accept: "application/json",
        "Content-Type": "application/json",
        },
        body: JSON.stringify({ prompt: input }),
    },
    );

Expected Behavior

How to use AI features on self-deployed services?

Environment

• Browser: Google Chrome 119.0.6045.199(arm64)
• Version: master branch

[dwelle]

Hi. The AI backend server is only meant to be used on *.excalidraw.com domains, and we will unlikely make it generally available (due to costs).

But you can deploy your own AI backend (and point the VITE_APP_AI_BACKEND to it). The response should resolve to { generatedResponse: string }.

We will be stabilizing and iterating on the AI API before the next stable release.

[rubickecho]

ok, very understanding, so it should be implemented by myself. can you provide a reference for its prompt? 👀 👀 👀

[bhNibir]

@dwelle Where can I find the AI backend source code?

I want to deploy the AI backend on my server. I need the Text to Diagram feature on my application.

[magedhelmy1]

@bhNibir can you please share the AI backend source code? Thank you

[bhNibir]

Hi @magedhelmy1 ,

I didn't find any AI backend source code. But I get answers from @dwelle on Discord. here is his message -

Hi. Not publicly accessible at the moment. It's part of our monorepo where we have Excalidraw+ code and such. But it's nothing super advanced currently. If you check out OpenAI docs for Completions API, you should be good to go! this is the gist

import OpenAI from "openai";

const generateResponse = async (prompt) => {
  const response = await openai.chat.completions.create({
    messages: [
      {
        role: "system",
        // something that tells GPT what it should do regardless of what user prompts
        content: `<SYSTEM PROMPT HERE>`,
      },
      {
        role: "user",
        content: `<USER PROMPT HERE>`,
      },
    ],
    model: "gpt-4-1106-preview",
  });
  
  const messages = response.choices.map((r) => r.message);
  return messages.map((m) => m.content).join("\n");
}

Here is my idea -

You can use OpenAI to get the mermaid diagram definition string and use mermaid-to-excalidraw to render in the editor.

See Mermaid to Excalidraw API

I hope this will help you. Thank you

[rubickecho]

ok, very understanding, so it should be implemented by myself. can you provide a reference for its prompt? 👀 👀 👀

This is my prompt demo

Create a Mermaid diagram using the provided text description of a scenario. Your task is to translate the text into a Mermaid Live Editor format, focusing solely on the conversion without including any extraneous content. The output should be a clear and organized visual representation of the relationships or processes described in the text.

Here is an example of the expected output:

graph TB
    PersonA[Person A] -- Relationship1 --> PersonB[Person B]
    PersonC[Person C] -- Relationship2 --> PersonB
    PersonD[Person D] -- Relationship3 --> PersonB
    PersonE[Person E] -- Relationship4 --> PersonC
    PersonF[Person F] -- Relationship5 --> PersonA
    PersonG[Person G] -- Relationship6 --> PersonF

[greycodee]

Why not open source text-to-diagram prompt? Is this a commercially available feature? @dwelle