☔️ When edit, etc. hits output token limits, output is truncated in edit and chat

[dominiccooney]

Version

Main 032645f0e65c3d7958ff9d8fb0f8b78c56091062

Describe the bug

This is an umbrella bug to collect a family of related issues with the same underlying cause.

Effect on edits:

• [ ] https://github.com/sourcegraph/cody/issues/3408
• [x] https://github.com/sourcegraph/cody/issues/3388

Effect on chats:

• [x] https://github.com/sourcegraph/cody/issues/3472
• [ ] https://github.com/sourcegraph/jetbrains/issues/1176

Expected behavior

• Edit should not truncate existing code when LLM output limit is reached. (Note, the typical use case can delete code, deleting trailing code looks like truncation, but that is the RIGHT thing to do in those cases.)

Exactly what we should do to communicate we hit a limit is unclear. @toolmantim, do you have ideas here?

Additional context

The underlying cause is: Edit, document, etc. work by having the LLM generate code. In some cases, re-generate input code with modifications. When that output is too large we can hit the token output limit of the LLM. Current behavior is to treat this as success, in the product UX and Telemetry. But that is wrong.