vscode-copilot-release
vscode-copilot-release copied to clipboard
microsoft
Copilot chat forgets previous response and often "starts from scratch" with follow-up questions
Something I've noticed about Copilot chat vs ChatGPT is the lack of conversation history awareness. ChatGPT keeps track of previous messages and replies in the context of the long-running conversation. Copilot constantly loses contextual awareness and as a result is pretty bad with follow up questions. It often replies as if it has no awareness of its previous response. But if you say "look at your previous response and try again" it gives you the answer you were looking for.
Please, please improve this. Copilot chat should always respond in the context of previous messages rather than the other way around.
FYI here is some info on how OpenAI does it with ChatGPT: https://community.openai.com/t/how-to-preserve-the-context-session-of-a-conversation-with-the-api/324986/3
In the website version of ChatGPT, the contextual continuity is achieved through the resending of all prior messages for each new communication, ensuring the conversation maintains its coherence.
To replicate similar functionality through the API, developers must structure their API requests as a sequence of messages within a conversation. Each message in this sequence encompasses a “role” (system, user, or assistant) and “content” (the text of the message). By incorporating all prior messages within the conversation history while making API requests, developers provide the model with the essential context to produce relevant responses.
Does Copilot chat resend previous messages with each request?
This is still an issue. I have to regularly do this to get Copilot to consider the previous message when replying:
This really shouldn't be necessary. It has a big impact on flow and efficiency.
Still an issue. Is this going to be resolved? If not don't think it is worth paying for
I'm facing the same problem. but i use PHPStorm + the official copilot Plugin. I think, the problem it's not related to the IDE, but part of the copilot itself.
The prompt often forgets the context I gave it before and responds as if it were restarting. This means that copilot cannot be used to analyze complex problems.
copilot chat did work well in the beginning and now it is unusable because of this problem.
FYI, it happens to me a lot more when I use @workspace in a followup message again. Maybe really just a context windows problem.
It got really bad for me in the past few days, I describe a specific problem, copilot asks me to provide a script for reference, but then explains the script instead of looking for the issue. If I then ask if he found the issue he asks me to provide the script again. i just tried it 8 times, different phrasing, trying to show the code in different ways. No luck
Commenting to keep this issue on the radar. I’ve noticed it’s getting better at understanding context across multiple files, but it has the attention span of a goldfish when it comes to follow-up questions.
Absolutely agree to all of the above, an infuriating problem and I hope github fix it quickly, has a time cost as well as an emotional one.
This issue keeps getting worse, today for the first time I couldn't even get Copilot to read previous replies stating that it was due privacy and confidentiality for the user. After 5 different prompts asking Copilot to re-read the chat it was impossible for me to make it work.
Yeah, I'm getting this issue, it completely forgot what we were working on. And I have to remind it with various prompts, I won't be paying for this.
I'm facing this issue as well. It NEVER keeps the prior questions in context in the chat (Not even the one asked immediately before the current one), therefore, follow-up questions are simply undoable. This was not the case when I started using it a few months ago. Very disappointing.
+1 I have been looking for a way around this since forever. I don't want to have to go outside the IDE to prompt ChatGPT despite paying for Copilot.
I face this problem and in fact when I started using it it followed the entire history
Same here. I end up uploading files into ChatGPT where it keeps the context and history instead of using copilot which defeats the main benefit of it!
I am also experiencing this issue. VSCode is the primary way that i utilize my copilot subscription. This bug makes the VSCode extension extremely unhelpful. I will be cancelling my subscription until this bug if resolved.
Why is this not yet fixed? This is a fundamental feature of any chatbot application. I have to unnecessarily keep adding previous chat information in new messages.
Please fix this.
Facing the same issue, not in vscode but in Copilot itself. It was not like this a few weeks ago. Clearly, they changed something. Rather than a bug, it seems like they're trying to cut costs by not processing prior information repeatedly. Regardless of the reasons, this is annoying 😒
This is still happening in GitHub Copilot Chat using GPT-4o.
https://github.blog/changelog/2024-07-31-github-copilot-chat-and-pull-request-summaries-are-now-powered-by-gpt-4o/
I guess it's probably a cost aspect of having to transfer all the old information. If so, it would be nice to have the ability to select a message from a previous answer and ask a question in the form of a quote or reference or reply.
By adding a reply button between those buttons for each message.
