CopilotForXcode icon indicating copy to clipboard operation
CopilotForXcode copied to clipboard

Copilot wrote codes to wrong files in Agent mode

Open owenzhao opened this issue 4 months ago • 11 comments

Describe the bug

Versions

  • Copilot for Xcode: 0.41.0
  • Xcode: 26.0 beta 5 (17A5295f)
  • macOS: 26.0 Beta (25A5338b)

Steps to reproduce When using Agent mode to fix bugs, sometimes Copilot will write code to the wrong file and make content duplicated.

For example, with a SPM project, Copilot wrote codes of Package.swift to swift file in sources. And it could not found this was bug until I pointed out.

It also happened with normal .xcodeproj project between .swift files.

When this happens, the whole file content will be replaced. You will lost one file and have two duplicated files.

Can you investigate and fix it?

Screenshots

Logs

Additional context

owenzhao avatar Aug 18 '25 04:08 owenzhao

Could you share the prompt and simple sample project to help reproduce it?

testforstephen avatar Aug 18 '25 05:08 testforstephen

I think the prompt is irrelevant. The prompt is simply "Fix bugs".

Here is the full story.

I am working on a backend server app based on Vapor. It uses SPM to manage the project. I add a user register router and need to import JWT framework.

Since there is not JWT in Package.swift, Copilot should fix this issue by adding it to Package.swift. However, there is a current file automatically chose and selected from Xcode. Copilot added JWT dependency to this file instead of Package.swift.

Then in Copilot's view, the JWT dependency was added. But for the swift build system, it was not.

What makes the things worse is that Xcode sometimes could give out some useful error information as "import PackageDescription" not working, as it should only existed in "Package.swift", not in other swift source files. But "swift build" that Copilot used can only told "JWT" model was not eixsted. So Copilot never knew it wrote to the wrong file.

owenzhao avatar Aug 18 '25 07:08 owenzhao

I finally workaround this by asking Copilot to recover current file and then open Package.swift in Xcode so it will be the current file in Copilot. Then I ask Copilot to fix bugs. This time it works.

owenzhao avatar Aug 18 '25 07:08 owenzhao

Since in Agent mode, Copilot can read the whole project. To fix this issue, maybe we could ask Copilot to work starting from the current file, but when editing, it should write to files that should be revised instead of more willing revise the current file.

owenzhao avatar Aug 18 '25 07:08 owenzhao

Today, I encountered another situation related to this issue. This was with an .xcodeproj project. At first, I was using GPT-4.1, but there was a problem it just couldn’t solve. So I switched to GPT-5. GPT-5 managed to solve the issue, so I continued using it.

For my new task, GPT-5 created a new view. However, it didn’t add this view to the project—meaning, the new file existed in the folder, but it wasn’t referenced in the .xcodeproj file.

I pointed out this issue, and it made the correction. But after the change, I discovered that not only did it modify the corresponding file reference in the .xcodeproj, it also changed the current SwiftUI view file to the content of the new file.

So, I told it to restore the current file to the correct content. It did restore it, but the file referenced under .xcodeproj was also restored. This shows that Copilot was conflating the two files; even the restoration was done together.

In the end, I chose to manually add this file to the project via Xcode’s menu.

owenzhao avatar Aug 19 '25 22:08 owenzhao

I've encountered the same problem several times. I haven't established a consistent pattern, but the closest I've come to establishing one is when trying to write a .entitlements file it writes the file to the .project file, corrupting the file/project completely. Fortunately since I've encountered this a few times I open the corrupted file and ctrl-Z to recover it (or recover from Git if ctrl-Z doesn't work). If I identify a clear pattern I'll document it, but I confirm I'm experiencing the same or a very similar issue to the one ownezhao has described.

vernonstinebaker avatar Sep 03 '25 06:09 vernonstinebaker

Did you use an OpenAI model as well?

owenzhao avatar Sep 03 '25 10:09 owenzhao

@owenzhao probably used GPT-5. But I've used other models too. I failed to make note of which model I was using when that happened, but GPT-5 is a strong possibility. I haven't been using the Anthropic models much at all, but I've used both GPT-5 and Gemini Pro 2.5 quite a bit.

vernonstinebaker avatar Sep 03 '25 10:09 vernonstinebaker

I think that was a vulnerability of OpenAI models, as I use GPT-5 and GPT-4.1. I didn't remember I had encountered this issue with Claude Sonnet 5. I think maybe OpenAI models can't understand Xcode projects and SPM projects well. Or Copilot used different methods/prompts to deal with those models, as OpenAI API and Claude API are different.

owenzhao avatar Sep 03 '25 10:09 owenzhao

I was wrong. I encountered this issue with Sonnet 4 either today.

owenzhao avatar Sep 06 '25 23:09 owenzhao

Just happened to me. Here's a screenshot of what should be a Swift file (it's actually the main entry point for the application) that's been overwritten by the project file that GitHub Copilot for Xcode was 'repairing'. I was using Gemini 2.5 Pro, so it doesn't seem like the issue is isolated to GPT-5. I've also attached the log file.

copilot.log.tgz

Image

vernonstinebaker avatar Sep 08 '25 11:09 vernonstinebaker