devika
devika copied to clipboard
stitionai
when building app with OLLAMA response returns Invalid response from the model, trying again...
Describe the bug when building app with OLLAMA as the model provider response returns Invalid response from the model, trying again... it seems to be stuck in a loop, i can see it writing code but itll just keep doing the same thing over and over, just using my local deepseek coder model
To Reproduce Steps to reproduce the behavior: start project, let it start progressing, eventually loops cant seem to get its self out
Expected behavior make the code? .
Screenshots
Desktop (please complete the following information):
- OS: windows
- Browser: firefox
- Version newest
Additional context Add any other context about the problem here.
Your model is likely not following the instructions well enough and doesn't return the expected format.
with hermes 2 pro 7b it seems to work best, even if the code it generates is not usable due to some syntax errors..
I have this similar error with dophincoder based on starcoder2 @jmtatsch
Your model is likely not following the instructions well enough and doesn't return the expected format.
can they be corrected with ollama modelfiles? if so where do I find the right parameters??
I don't know any local models that are good enough yet to work for this usecase. Maybe mixtral based ones but I am to gpu poor to try that out.
Your model is likely not following the instructions well enough and doesn't return the expected format.
A basic question: Is there ANY open source LLM that can be used in this project, instead of GPT-4?
I also played with gpt-pilot and it has the identical problem, works with GPT-4 decently, but any open source model I tried, didn't work.
with hermes 2 pro 7b it seems to work best, even if the code it generates is not usable due to some syntax errors..
I have this similar error with dophincoder based on starcoder2 @jmtatsch
Your model is likely not following the instructions well enough and doesn't return the expected format.
can they be corrected with ollama modelfiles? if so where do I find the right parameters??
Not likely. You are basically asking if it is possible to create a system prompt for a much smaller model to behave like GPT-4.
If it was this easy then no one would be using OpenAI API at all.
ah i see! i guess id just hoped that if it had support something would be runable to the point it could be used, as tech gets better im sure we will get there
Not likely. You are basically asking if it is possible to create a system prompt for a much smaller model to behave like GPT-4.
If it was this easy then no one would be using OpenAI API at all.
Hermes-2-Pro-Mistral-7B it's not that bad, he creates a little bit of code and it also seems to make sense, he mostly makes syntax errors (I asked him to make the first level of super mario bros and he went to analyze the various sites and then built something in python in pygame with all files and folders sorted in the project)
Ok, but the system itself is designed to correct syntax errors through more iterations. Add prompting to correct its own mistakes.
On Fri, Apr 5, 2024, 11:11 AM nokirunner @.***> wrote:
Not likely. You are basically asking if it is possible to create a system prompt for a much smaller model to behave like GPT-4.
If it was this easy then no one would be using OpenAI API at all.
Hermes-2-Pro-Mistral-7B https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B it's not that bad, he creates a little bit of code and it also seems to make sense, he mostly makes syntax errors (I asked him to make the first level of super mario bros and he went to analyze the various sites and then built something in python in pygame with all files and folders sorted in the project)
— Reply to this email directly, view it on GitHub https://github.com/stitionai/devika/issues/347#issuecomment-2040052403, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDD3ZO5VCYVU6CU32RTFDTY325KDAVCNFSM6AAAAABFXBCXJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQGA2TENBQGM . You are receiving this because you commented.Message ID: @.***>
I am using gpt-pilot with ollama serving Llama 3 70b Instruct model. Seems to work in general, but I am not sure yet if the code it produces is any good. It creates files however, so function calling seems to work.
On Sat, Apr 20, 2024, 3:03 AM mahiatlinux @.***> wrote:
Any updates or work arounds? I want to try the new LLAMA 3 8B.
— Reply to this email directly, view it on GitHub https://github.com/stitionai/devika/issues/347#issuecomment-2067583932, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDD3ZJRO6LH5Y42NLZNX3LY6IHN5AVCNFSM6AAAAABFXBCXJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXGU4DGOJTGI . You are receiving this because you commented.Message ID: @.***>
fresh install today with usage of llama 3 8b, the error still persist (win 10, anaconda, python 3.10)
@ARajgor
with the "codeqwen" model (It works with other models too)
devika performed the tasks but instead of giving me the code it gave me the example in the agent coder prompt as a result (every time the same result, also whit other ollama models). I then went to look at the agents, and I modified the coder prompt, and it finally worked, I didn't get working code, but the model executed the tasks correctly.
devika\src\agents\coder\prompt.jinja2 I added to this piece of prompt: "Your response should only be in the following Markdown format" "like this example obviously replacing the example code with your own code:"