vscode-gitlens icon indicating copy to clipboard operation
vscode-gitlens copied to clipboard

Improve experience for invalid or poor generateRebase responses

Open d13 opened this issue 8 months ago • 0 comments

Missing or incorrect hunks

On rare occasion, the AI provider responds with commits that:

  • don't include all the hunks we sent
  • include hunk numbers we didn't send
  • use the same hunk twice

Our current short-term solution is to provider an error message to the user that explains the AI response was missing changes and to retry again.

Ideally, a better retry solution would be to continue the conversation with another turn, telling the provider that it missed some changes, provide the bad response they sent and ask it to fix it.

Poor responses

We should have an evaluation solution to help determine poor responses before the user does and use another turn to retry and improve the quality of the response.

d13 avatar Jun 13 '25 21:06 d13