CopilotForXcode icon indicating copy to clipboard operation
CopilotForXcode copied to clipboard

Context is not clear but shared between tabs

Open owenzhao opened this issue 6 months ago • 5 comments

Describe the bug

People create new tab to get fresh ideas. But currently, Copilot shares the context and leads errors.

Versions

  • Copilot for Xcode: 0.37.127
  • Xcode: 16.4 (16F6)
  • macOS: 15.6 Beta(24G5054d)

Steps to reproduce

  1. Ask copilot to change code for some times, for example, 4 times.
  2. Use GitHub to restore to the original state.
  3. Open a new tab, ask the first question.

Expected: Copilot only dealt with the specific question.

What happened: Copilot revised the code with the four questions. Even if you change the model, for example, from Sonnet 4 to Gemini 2.5 Pro, the result was the same.

Screenshots

Logs

Additional context

owenzhao avatar Jun 30 '25 09:06 owenzhao

@owenzhao this is an interesting issue. I try to repro the issue, here are steps I used:

  • New a chat tab, ask some question about current file
  • Then new another tab, but disable current file, and ask "summarize chat history" there
  • The AI told me it cannot see previous chat history

This reflects the chat history is not shared across tabs. Do you have snippets to consistently repro the issue? thanks.

testforstephen avatar Jun 30 '25 10:06 testforstephen

Let me talk about how I made my judgment.

I'm working on an application for forced breaks. When the time is up, it will force the user to take a break. This requires a screensaver-like view. If the user wants to leave early, they need to enter a password.

I chose Sonnet 4. The AI quickly wrote the initial version. Then I ran a test. During the test, I found that although the screensaver could run in full screen, if the user also had other full-screen applications, such as a video application running in full screen, the forced break would only run in the background and not switch to the foreground.

So, I added the requirement to obtain focus. After that, the nightmare began. After adding this requirement, it could indeed actively obtain focus. But every time after entering the password, the screensaver would reopen immediately when exiting. And the countdown reset. It could only wait until the countdown ended and closed by itself, and the application would crash after the countdown ended. In addition, if you use gestures to switch desktops, multiple password input windows would appear. These were all caused by the screensaver application creating too many redundant NSWindows.

After several more rounds of conversations. I found that the AI itself really couldn't solve this problem. So I decided to solve it manually. So I asked the AI to change the full-screen mode of the screensaver to windowed mode in debug mode, and keep the rest unchanged. This would make it convenient for me to debug using Xcode. The AI did as I asked. But I found that in windowed mode, it seemed that there were no such problems mentioned before.

So I had to cancel all the modifications in GitHub Desktop. Then I created a new tab in Copilot. I asked a new question, which was similar to the initial question. When I ran the code modified by the AI, I found that this time the AI directly added Debug-related content in full-screen mode, which was exactly the same as the Debug interface of the window I mentioned last time. And I didn't mention anything about Debug this time at all. Moreover, this time it directly implemented a lot of things I didn't mention at all, that is, some of the problems I mentioned before. Those problems were all carried over. It was as if I didn't open a new tab but continued the conversation in the previous one.

I thought it was a problem with Sonnet 4. It might have remembered my conversation. So I opened another tab, chose Gemini 2.5 Pro, and asked the same question. As a result, the code given by Gemini 2.5 Pro was extremely similar to the previous code of Sonnet 4 and also included the Debug interface. Therefore, I concluded that it was a problem with Copilot.

owenzhao avatar Jun 30 '25 11:06 owenzhao

After seeing your reply, I plan to use the history function of Copilot and take a screenshot of our conversation to show you. However, I found that the history of the conversations before and after are all there, but the ones with problems are missing. I wonder if it has something to do with my canceling the modifications to those codes in git.

owenzhao avatar Jun 30 '25 11:06 owenzhao

Ah, I see, thanks for the explanation. This is likely a local file sync issue. When you revert some local modification via GitHub Desktop, somehow Xcode Copilot didn't detect the update. As a result, when you started a new chat, it fed the AI model the version of the file from before the revert.

testforstephen avatar Jun 30 '25 12:06 testforstephen

What is the workaround? Roll back code in Xcode instead of GitHub Desktop?

I know quit Copilot and restart must work. But some times this will fails in some way and request to restart Xcode.

owenzhao avatar Jun 30 '25 17:06 owenzhao