vscode-copilot-release
vscode-copilot-release copied to clipboard
microsoft
generating edits so slow with Github Copilot Edits
Type: Performance Issue
generating edits so slow vscode
Extension version: 0.23.2 VS Code version: Code 1.96.1 (Universal) (42b266171e51a016313f47d0c48aca9295b9cbb2, 2024-12-17T17:50:05.206Z) OS version: Darwin arm64 23.6.0 Modes:
+1 it would write "Generating edits..." and just get stuck there. Slow in this case, would be an improvement!
It seems to get slower the more editing sessions it does. Maybe there's a bug in how the context is generated for the API call for Claude. Specifically, the slow part seems to be "Applying edits" for me.
For the generation step being slow, we made some improvements here in December, could you please try using http://code.visualstudio.com/insiders/ and the latest prerelease of Copilot Chat? If you have a prompt and some files where you reliably see slowness, that would help.
@mbtx2 the applying edits phase being slow is separate, can you please file a new issue share more info on what files you're editing (language and length) as well as the nature of the changes?
@joyceerhl I'm running VS Code insider (Version: 1.97.0-insider Commit: 89f808979a5151bd91324e65d4f7ab1b62896983 Date: 2024-12-20T05:04:19.167Z (2 wks ago)) with copilot pre-release copilot (0.24.2025010701) and I still hit very slow edit. There is no way to find a set of file or a specific prompt. It just happen randomly (and file size seems to have no impact). Some time it works ok (still much slower than Cursor: https://x.com/cursor_ai/status/1874082034928472393) but other time it barely works or simply never apply the change. It seems like dependant on the load on your servers or something like that. I'm unsure why it is so bad to be honest. If there is a way to generate logs, let me know, I will enable it and share the logs when it happens again. But even when it works, it's still pretty slow. There is multiple review of Copilot Edit on YouTube and they all share the same experience with generation that never complete or that take forever, so it's a widespread issue.
We recently made improvements here in the client and the service for generating edits, @hmorneau could you please update to the latest Insiders and give it a try there?
We recently made improvements here in the client and the service for generating edits, @hmorneau could you please update to the latest Insiders and give it a try there?
@joyceerhl Wow, ok, much much faster now. Can you explain the change. What ever you did it's very good now.
Thanks for the improvements @joyceerhl. The issue I see now is the time it takes to start the file editing. I use Claude 3.5 and the Agent Mode in the lastest Insiders version.
So, it writes "Generating..." and then it takes sometimes 10-20 seconds before editing. To give you a comparison, the vs code extension Cline that I also use is almost instant. It doesn't add to the latency of the model (Claude 3.5).
The work you did on the UX/UI integration with VS Code is pretty neat, I just wish that extra latency was reduced to the max!
Thank you, Danny
@ferrind Do you see the same for GPT-4o? If no, this could indicate a service issue
@joyceerhl I didn't really try GPT4-o or Gemini Flash. I pretty much use Claude 3.5 most of the time. Is it possible for you to check if that latency is different for different AI models providers?
It is, that is why I ask. There was some improvement on the service side recently, but it sounds like there may have been a regression
@joyceerhl I did try different model and it doesn't seems to have any impact on the speed of the apply. Sometime it's really fast, other time a bit slower to get going and it will apply slower as well. Nothing as bad as when this issue was first reported. What ever you did, it improved things a lot. There is probably more optimization to do, but overall it's decent.
Maybe not a regression, maybe the service is just more popular now that Copilot is really getting great. Maybe look at the load of the service. If you have the period of the day that it's more quiet vs peak time, I can do test during those time to compare.
@joyceerhl I was able to capture it:
Model 4o 2025-02-13 14:45:43.484 [info] Fetched model metadata in 497ms 6287120a-dde7-45d5-915a-5bea2627f600 2025-02-13 14:47:40.265 [info] message 0 returned. finish reason: [stop] 2025-02-13 14:47:40.267 [info] request done: requestId: [7fd793e3-3e92-4fb9-8497-cfdc0be39794] model deployment ID: [] 2025-02-13 14:49:12.633 [info] srequest done: chatRequestId: [7fd793e3-3e92-4fb9-8497-cfdc0be39794], speculationRequestId: [0b9dc92e-6e62-43d6-a572-754456394df7]
14:45 I submitted the edit request. 14:47 the generating finally started 14:49 the generating completed So it took 2 minutes to get start and nearly 2minutes to complete the edit
Sonnet 3.5 (Exact same request) 2025-02-13 14:53:06.073 [info] message 0 returned. finish reason: [stop] 2025-02-13 14:53:06.074 [info] request done: requestId: [b217ec5f-2b6d-4f02-bed0-12de0e014858] model deployment ID: [] 2025-02-13 14:54:14.049 [info] srequest done: chatRequestId: [b217ec5f-2b6d-4f02-bed0-12de0e014858], speculationRequestId: [07bd00e8-a90f-461b-ba2e-db10a48d8326]
The time to start generating was much faster, but the apply was above 1 minute in this case. The apply speed doesn't seem to depends on the model used.
I hope it helps.
I really believe there's a lot of variation in the generating step regardless of models, if that's the case, it would be interesting to understand why @joyceerhl.
Again, if you guys can cut that generating time, you will have a hell of a product. I guess, the more you will be closer to "metal" which is the LLM itself, the faster it will become.
Regards, Danny
@joyceerhl it's getting really slow again. Cursor came with their own model that process 5000token/sec, which make edit instant. Are you coming with something like this?
it can be sometimes slow and sometimes fast regardless of the complexity of the edit and the number of files involved, im using consistently 3.7,
I have asked claude to explicitly provide +- diff without creating any gaps when quoting code to be deleted/added (having gaps confuses github copilot a lot, especially when editing readme-like files with quotes). I missed line numbers but otherwise this is a one second apply diff scenario, while it takes minute(s) using github.
4o is not much better. Simply "applying edits" feature is slow. This is more of a feature request, but maybe it would be better to provide consistent format for diffs which could be applied by user? Roo Code connected to github copilot model (4o) is generating diffs much faster (almost instantenous).
It's really slow specially because it generate and rewritte the file from scratch.
same here i could write a book with no fingers quicker than claudes editing
Hey folks, we're working on an improved fast apply experience that shows promising speed ups we hope to deliver very soon. Stay tuned for updates!
Hey folks, we're working on an improved fast apply experience that shows promising speed ups we hope to deliver very soon. Stay tuned for updates!
@pierceboggan Thanks, that's currently the most painful part of the experience of using copilot. How soon can we test this out? Any hidden flag we can enable now :)
how to cancel/terminate applying edits when it's taking forever.
#5611
But that would a waste of time to implement this. The real solution is to have a custom model that do those apply. Cursor came up with a model that process 5000 token/sec to do that. So all the apply are instant. I'm curious to see what @pierceboggan is cooking.
It was a bit faster, and now it's slowing down again. I had time to search for this topic, post this reply and the edit is not done yet.
edit: Still ongoing 87%
While I am still subscribed, I had to switch to using Cursor. I have faith Github copilot would catch up and get all the goodies of cursor. keep us posted