gitdiagram
gitdiagram copied to clipboard
ahmedkhaleel2004
Syntax error in text mermaid version 11.4.1
In the step 3/3, "Syntax error in text mermaid version 11.4.1" occurs frequently. Maybe we should state In the prompt that the LLM is required to strictly adhere to the Mermaid syntax.
This error simply means the LLM generated incorrect Mermaid.js diagram code.
I've tried addressing this issue in numerous ways but ultimately I find that there is no reliable way. It is mostly a limitation of the LLM. If there is a way to validate Mermaid.js code, that would help but as of now I am not sure how.
Maybe we should state In the prompt that the LLM is required to strictly adhere to the Mermaid syntax.
I already do this, for the errors that I've specifically found. You can see it in the prompts.py under the third prompt (that generates the Mermaid.js code), but of course this is not guaranteed and new errors always show up.
This error simply means the LLM generated incorrect Mermaid.js diagram code.
I've tried addressing this issue in numerous ways but ultimately I find that there is no reliable way. It is mostly a limitation of the LLM. If there is a way to validate Mermaid.js code, that would help but as of now I am not sure how.
Maybe we should state In the prompt that the LLM is required to strictly adhere to the Mermaid syntax.
I already do this, for the errors that I've specifically found. You can see it in the
prompts.pyunder the third prompt (that generates the Mermaid.js code), but of course this is not guaranteed and new errors always show up.
Thanks for the reply. I've used this before, but it was a few months ago and it worked fine. I fully understand why it might take time (or may never get fixed) but thank you!
That's an LLM issue; most LLMs still can't generate an error-free Mermaid Diagram.
I suggest, as a fallback, that you return to the user the Incorrect Code, The user could tune it or give it back to another LLM to tune and fix it. That will be beneficial
@ahmedkhaleel2004
That's an LLM issue; most LLMs still can't generate an error-free Mermaid Diagram.
I suggest, as a fallback, that you return to the user the Incorrect Code, The user could tune it or give it back to another LLM to tune and fix it. That will be beneficial
Opened #34, @ahmedkhaleel2004. Can you please check it out?
Is there a way for the mermaid.js code to be ran through a separate bash process where code valdiation is determined by the exit code and the llm can self-correct based on the error stack? (W/out having taken a look at the current repo structure, Im assuming that this is a possibility)
Hey may I know what LLM you r using? Mistral AI codestral one generates error free mermaids most of the time. I used the same model in one of my app code to flow charts. And it's dam cheap aswell
what LLM you r using
openai o3-mini
It might be worth giving Gemini a shot with mermaid documentation in cache. The large context window usually solves these issues.
I tried with two of my repos written in Python. It worked when I stayed on the tab without switching, but when I tried it and I changed the tab during the diagram creation it failed
I tried to get as much info as I could on this issue by looking at the doc and testing on my side (https://mermaid.js.org/intro/syntax-reference.html)
- small repository seems to be ok
- medium repo has trouble if files are long, it get lost easily
- large repo has 100% chance of not displaying
no matter the custom instruction, we can't influence the 3rd step before the rendering... the custom instruction only seems to affect the 1st step
Adding a custom instruction like "Fix syntax error" fixes the mermaid errors for me
+1
This repository : https://github.com/HeroTransitions/Hero
In step 3 also throw this Exception
