vscode-copilot-release icon indicating copy to clipboard operation
vscode-copilot-release copied to clipboard

Agent mode claimed to be making edits, did not make edits

Open 333fred opened this issue 8 months ago • 2 comments

Type: Bug

Agent mode repeatedly claimed to be making edits to a function in the file. It repeatedly failed to do so, recognized that it failed, and then failed again.

Extension version: 0.26.2025040204 VS Code version: Code 1.99.3 (17baf841131aa23349f217ca7c570c76ee87b957, 2025-04-15T23:18:46.076Z) OS version: Linux x64 6.8.0-58-generic Modes:

System Info
Item Value
CPUs AMD Ryzen 9 5950X 16-Core Processor (32 x 4674)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: disabled_software
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off
webnn: disabled_off
Load (avg) 1, 1, 3
Memory (System) 62.70GB (37.25GB free)
Process Argv --crash-reporter-id c9f05a69-d019-464b-b1f2-b43e5fb75046
Screen Reader no
VM 0%
DESKTOP_SESSION regolith-x11
XDG_CURRENT_DESKTOP Regolith:GNOME-Flashback:GNOME
XDG_SESSION_DESKTOP regolith-x11
XDG_SESSION_TYPE x11
A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805cf:30301675
binariesv615:30325510
14424c2-chatv2:31289885
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
2e7ec940:31000449
pythontbext0:30879054
cppperfnew:31000557
dwnewjupyter:31046869
pythonrstrctxt:31112756
nativeloc1:31192215
5fd0e150:31155592
dwcopilot:31170013
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
hdaa2157:31222309
copilot_t_ci:31222730
e5gg6876:31282496
pythoneinst12:31285622
bgtreat:31268568
4gafe986:31271826
31787653:31262186
3e8i5726:31271747
996jf627:31283433
useunpkgapi:31288273
7bj51361:31289155
747dc170:31275177
aj496949:31278748
aj953862:31281341

333fred avatar Apr 24 '25 17:04 333fred

Here are the screenshots from the interaction. None of these edits that it talked about actually appeared. Additionally, rather than editing all of the tests in the file in the same edit, it persisted in doing a single method at a time, taking quite a while before I canceled it.

Image Image

333fred avatar Apr 24 '25 17:04 333fred

happens on 25% of my requests. at least put in a simple check to see if the file was actually edited so you do not charge us for the failed request.

p3nGu1nZz avatar Apr 28 '25 12:04 p3nGu1nZz

Which model is this? Can you check for errors in the Copilot Chat output channel?

roblourens avatar May 05 '25 17:05 roblourens

This was with Sonnet 3.7. Far too much time has passed for me to be able to say anything about errors.

333fred avatar May 05 '25 20:05 333fred

Thanks. Sometimes this kind of loop happens due to an error in using the tool. We may have fixed an issue with tool calling for Claude in the meantime (server-side or in Insiders/this week's release). Sometimes it happens because the model is not smart. If you see it again, would be helpful if you can see whether it ran into an error.

roblourens avatar May 05 '25 22:05 roblourens

Image

It only did the first bullet of these edits. There was nothing that I could see in the chat log for errors:

2025-05-06 14:20:56.049 [info] Fetched model metadata in 624ms 63c19b26-ada9-40d8-913f-4b2b38407ea5
2025-05-06 14:20:56.663 [info] Fetched content exclusion rules in 341ms
2025-05-06 14:21:05.841 [info] message 0 returned. finish reason: [stop]
2025-05-06 14:21:05.841 [info] request done: requestId: [8025ce08-01f6-4634-861c-e9e802b4dc05] model deployment ID: []
2025-05-06 14:21:08.152 [info] message 0 returned. finish reason: [stop]
2025-05-06 14:21:08.152 [info] request done: requestId: [aee7b818-efcc-4a47-8f5c-277889d84eae] model deployment ID: []
2025-05-06 14:21:08.156 [info] srequest done: chatRequestId: [8025ce08-01f6-4634-861c-e9e802b4dc05], speculationRequestId: [8c2662c4-9805-4df5-9d34-6a10b8592bf2]

333fred avatar May 06 '25 21:05 333fred

Usually Claude is smarter than this ☹

roblourens avatar May 06 '25 22:05 roblourens

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!