opencode icon indicating copy to clipboard operation
opencode copied to clipboard

gemini doesn't handle edit tool very well

Open adamdotdevin opened this issue 6 months ago • 29 comments

it fails a lot with "oldString not found in file. Make sure it matches exactly, including whitespace and line breaks". could be something we could fix with whitespace normalization?

adamdotdevin avatar Jun 20 '25 17:06 adamdotdevin

It also happened quite frequently in the old, we don't talk about anymore, opencode. It worked slightly better with patch, but at around 70k+ token context, it fails quite persistently on it and edit.

We actually have quite a few options: Aider has diff-fenced, avante.nvim has a poor man's cursor apply, and one guy made CEDARScript for Gemini back before it was cool. I think the core elements in CEDARScript could be something we can adapt for our case; like offloading precise line numbers, indentation counts, add whitespace to that, and syntax consistency to a runtime.

To add on, Gemini 2.5 Pro/Flash also worked with a lot less errors on Cline's XML tool calling, not sure whether we should consider that in the near future.

monotykamary avatar Jun 21 '25 18:06 monotykamary

it fails a lot with "oldString not found in file. Make sure it matches exactly, including whitespace and line breaks". could be something we could fix with whitespace normalization?

I experimented a while back with fixing this issue with Claude code by creating my own mcp server edit tool. It was quite hacky but ultimately the two conclusions I came to:

  1. As you alluded to @adamdotdevin the most success I found was with simply scrubbing the indentations (both in kind like tabs vs spaces, and in size like 2 vs 4 spaces). I used the "trimming" as a way to match / find the old content but then would still replace correctly with the real new content with actual indents provided by LLM.

  2. I am not sure if you use line numbers at all with the opencode edit tool but if you do, LLMs are horrid at line numbers. They often skip over empty lines so the line numbers they consider are off kilter. If you use line numbers at all, I suggest just using it as a safety net in case there are multiple matches in the same file of the old content string, but if only one is found anyways, the line numbers don't matter so just replace it. Or maybe you don't use line numbers and this doesn't matter.

Hope this helps!

GitMurf avatar Jun 22 '25 15:06 GitMurf

@GitMurf so i'm going to implement an alternate edit tool for gemini - we have the ability to swap tools out automatically per provider

can you explain more what you mean by scrubbing the indentations? like you'd trim every line in the file, old string, new string and do the replace?

thdxr avatar Jun 22 '25 16:06 thdxr

can you explain more what you mean by scrubbing the indentations? like you'd trim every line in the file, old string, new string and do the replace?

Great question! Sorry I wasn't very clear. If LLM passes back an "old string" to match on to replace of the following:

    const foo = "bar";
    const bar = "baz";

Then instead of looking for that string with the indentation, I would look for the following:

const foo = "bar";
const bar = "baz";

Now as you alluded to, you'd essentially have to trim the entire file indents too if you wanted to try and do a full string match on this successfully. But I thought that was excessive. So the way I did it is I start by iterating through the lines of the real file, trimming each line as I check it and look for a match to the first line provided by the LLM (const foo = "bar";) ... if I find a match at line 27 then I try matching lines 2...n (in my example above it would just be matching the next trimmed line of const bar = "baz";).

If there happened to be two of the same old strings in the file, this also by default would replace the first found occurrence so that you can instruct the user this is the expectation in the rare case they have the same content multiple times in the file.

TLDR:

  1. Split the old string match coming from LLM into array of lines.
  2. Trim the indentations from incoming LLM old string lines.
  3. Loop through real file lines looking for a match to the first line coming from LLM (trimming to match).
  4. When first line is found, then loop through the other lines provided to see if they also match trimmed.

The other thing to note is I actually instructed the LLM to provide its old content and new content info each as an array of lines to avoid needing to split the content string by line break.

Also I first actually checked for exact match and edited the file with exact match if found. I only did the trimming logic as a fallback if a match wasn't found.

Does this make sense? Let me know if you have any questions and also would be happy to review any code you put together.

GitMurf avatar Jun 22 '25 16:06 GitMurf

Also @thdxr fyi it may be useful to allow the user to choose the tool to use as for me, gpt-4.1 also has been having some issues with the old string matching with edit tool so I could see other non Gemini models also want to still try this other implementation of edit tool.

GitMurf avatar Jun 22 '25 16:06 GitMurf

hey, i am having the same old string issue consistently when using ollama/qwen3. so it(s not just a gemeni thing

moda20 avatar Jul 05 '25 20:07 moda20

perhaps it'd be good to implement a separate edit model that the user can choose. One model for the reasoning and coming up with changes, and another model for the sole purpose of applying the edit.

Then let the user choose what model to use there (i.e. self-hosted fastapply, morph fast apply, relace instant apply, llama3.3-70b, or any other model) and make it do whole or diff edits similar to what aider does?

BigMitchGit avatar Jul 05 '25 20:07 BigMitchGit

I can also confirm that while using both ollama and llama.cpp with qwen2.5-coder and qwen3 that I'm also constantly getting this "Error: oldString not found in content or was found multiple times" (or errors extremely similar to this; I've seen the one posted by OP as well).

JamesMowery avatar Jul 05 '25 21:07 JamesMowery

I'm using Claude Sonnet 4, and getting this regularly: Error: oldString and newString must be different

codemonkey76 avatar Jul 16 '25 00:07 codemonkey76

@codemonkey76 this also happens with local, and gemeni, models too

moda20 avatar Jul 16 '25 11:07 moda20

got this error today with grok/kimi-k2 it actually went into an infinite loop sending the same payload to the llm again and again, I stopped it after 20 times

same tool call and string:

Error: oldString and newString must be different

moshemarciano avatar Jul 24 '25 12:07 moshemarciano

I can reliably trigger this error with a variety of models; I don't think it's provider specific.

esafak avatar Jul 25 '25 23:07 esafak

This seems to be all I'm getting on macos. Model tries to edit, this errors pops up, makes the same text insertion and starts all over. Edit: with chatgpt o4-mini

benmerckx avatar Jul 30 '25 10:07 benmerckx

I suggest trying square-bracket tool call format from aider pr 3781 https://github.com/Aider-AI/aider/pull/3781

It uses a format like this (snippets from my project)

[tool(ToolName, parameter="value", another_parameter="value")]
[tool(AnotherTool, param1="value1", param2="value2")]

with complex arguments enclosed in triple double quotes:

  • Arrays and objects: wrap JSON payload in triple quotes: """[...]""" or """{}"""
[tool(TodoWrite, todos="""[{"id": "2", "content": "Update config", "priority": "high"}]""")]

I just implemented this in my anthropic compatible proxy for claude code, and now Gemini can consistently perform one-shot edits containing literal escape sequences (like \n) with ease.

it still doesn't solve the uniqueness issue, but it addresses the biggest problem with gemini, which is escape sequence handling (\n \t \r etc) https://github.com/google-gemini/gemini-cli/blob/3a3b1381950fb3aab09f71e0ad1662a4f77b3c43/packages/core/src/utils/editCorrector.ts#L704

nyawox avatar Aug 07 '25 21:08 nyawox

I'm not sure if this is oldString/newString-related, but just since recently when switching from "plan" mode to "build" mode (Gemini Pro 2.5), I get this all the time:

Edit

The arguments provided to the tool are invalid: Model tried to call unavailable tool 'edit'. Available tools: invalid, bash, webfetch, glob, grep, list, read, todowrite, todoread, task.

If hitting /new followed by /session and selecting my session, Gemini Pro 2.5 can continue with editing files again in build mode.

fredrikaverpil avatar Aug 08 '25 20:08 fredrikaverpil

Auto-formatting code after the LLM outputs it will obviously break the replace-based editing in case the LLM's code style does not match the formatter's. (since now the context has different code than the file)

Even Anthropic's documentation makes this fatal suggestion - add a hook with a formatter... immediately after adding that, Claude Code's own edit tool started failing much more often.

🤦

jaens avatar Aug 09 '25 19:08 jaens

I have been getting this error frequently with Grok Code Fast 1: Error: oldString not found in content

OscSer avatar Sep 11 '25 21:09 OscSer

I have been getting this error frequently with Grok Code Fast 1: Error: oldString not found in content

Same here too, grok often fail to work with YAML file on my side for the indentation. It tries to reiterate the fix but never succeeded to fix it. It is not all the time, but it is often enough.

Tsirimaholy avatar Nov 07 '25 16:11 Tsirimaholy

It seems that since i migrate to v1 (actually v1.0.81) , Big pickle or Grok fast 1 are easily stuck in loop with this error , i m not reminding this so much in 0.8

Image

pleclech avatar Nov 20 '25 20:11 pleclech

It seems that since i migrate to v1 (actually v1.0.81) , Big pickle or Grok fast 1 are easily stuck in loop with this error , i m not reminding this so much in 0.8 Image

Can confirm, I seem to get stuck here enough to make this unusable for Big Pickle at least

Eduard-Gyarmati-Kibo avatar Nov 21 '25 14:11 Eduard-Gyarmati-Kibo

Since yesterday I have go back with 1.0.5 (from 1.0.81) and for the moment have not caught it in hell loop.

pleclech avatar Nov 21 '25 14:11 pleclech

I don't think we have changed the edit tool in months so Im not sure if that would be the issue..

rekram1-node avatar Nov 23 '25 20:11 rekram1-node

Still got "oldString and newString must be different" infinite loop. I tried using GLM-4.6 and DeepSeek.

narzokane avatar Nov 24 '25 17:11 narzokane

I'm having the same "oldString and newString must be different" and "oldString not found" errors on Ollama with gpt-oss:20b

Edit: This happens in 1.0.108 and 1.0.0.109, but NOT 1.0.85

catthetech avatar Nov 24 '25 22:11 catthetech

Could it be related to the format of read tool?

Results are returned using cat -n format, with line numbers starting at 1

So LLMs don't get file content directly and have to reformat it to supply it to edit tool. I constantly have problems with indentation in edits which breaks languages with significant indentation.

EDIT: tried to remove line numbers completely, and so far have much better results with Devstral.

kurnevsky avatar Dec 22 '25 14:12 kurnevsky

I'm still getting this with opencode v1.0.202 and gemini 3 flash.

mvelbaum avatar Dec 26 '25 11:12 mvelbaum

opencode is pretty much unusable once you reach over a certain amount of context, every code search or whatever it's doing fails with oldString not found in content

copilotenstar avatar Dec 27 '25 12:12 copilotenstar

getting the same issue with GLM-4.7

ImLunaHey avatar Dec 27 '25 16:12 ImLunaHey

DCP plugin and instruct the agent to verify changes were made before proceeding

tadghh avatar Jan 03 '26 03:01 tadghh