llm-code-interpreter icon indicating copy to clipboard operation
llm-code-interpreter copied to clipboard

support for unified diff or git diff format

Open unicomp21 opened this issue 1 year ago • 13 comments

otherwise we burn large token counts when doing large refactors?

unicomp21 avatar Jul 30 '23 21:07 unicomp21

Hi @unicomp21, if I understand correctly you want an alternative to the "WriteFile" operation that is not writing the whole file every time, but just applies the edits in some diff format?

ValentaTomas avatar Jul 30 '23 21:07 ValentaTomas

yes

unicomp21 avatar Jul 30 '23 21:07 unicomp21

I'm kinda wondering if "language server protocol" might be useful here as well? (ie deno lsp, typescript lsp, might also help drive down token counts during refactor operations)

unicomp21 avatar Jul 30 '23 21:07 unicomp21

I understand, maybe there is something there. We have been actually running language servers in these envs in the past, but for other reasons.

ValentaTomas avatar Jul 30 '23 22:07 ValentaTomas

Looking at unified diff in openai evals, looks like we need a different/simpler approach for handling code deltas? I wonder if gpt-4 could supply simple search/replace string? along w/ target filename?

unicomp21 avatar Jul 31 '23 12:07 unicomp21

You can probably start experimenting with it right now - tell it to use RunProcess to modify files by executing commands that apply the diffs. Then we can see if it can handle it somehow.

ValentaTomas avatar Jul 31 '23 13:07 ValentaTomas

Will do, I just applied for developer access to chat plugins, I'm on the waitlist.

I've been experimenting w/ this problem of eliminating cut/paste for a while. Here's an old eval of mine. https://github.com/openai/evals/pull/771

Starting to think sed w/ search/replace might be the simplest/reliable way for handling deltas.

using the new "custom instructions", this seems to work nicely, gpt-4 appears to be comfortable base64 encoding the deltas.

{ "deltas": [ { "type": "code_diff", "search_string": "cHJpbnQoIkhlbGxvLCBXb3JsZCEiKQ==", "replace_string": "cHJpbnQoIkhlbGxvLCBXb3JsZCEiKQoKcHJpbnQoIkdvb2RieWUhIik=", "file_path": "main.py" } ] }

unicomp21 avatar Jul 31 '23 13:07 unicomp21

Lol, I look forward to a setup w/ zero cut/paste. I'm lazy !!!

unicomp21 avatar Jul 31 '23 13:07 unicomp21

@ValentaTomas how long of a wait does it take on the plugins developer waitlist?

unicomp21 avatar Aug 01 '23 12:08 unicomp21

I don't really know.

We actually don't have and developer access to plugins and developed this with the help of the community by remotely debugging it.

ValentaTomas avatar Aug 01 '23 12:08 ValentaTomas

Not sure I understand, how would I do that?

On Tue, Aug 1, 2023 at 7:47 AM Tomas Valenta @.***> wrote:

I don't really know.

We actually don't have and developer access to plugins and developed this with the help of the community by remotely debugging it.

— Reply to this email directly, view it on GitHub https://github.com/e2b-dev/chatgpt-plugin/issues/1#issuecomment-1660244999, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFL7LOWT6GPIRHIDWR2CTXTD3HFANCNFSM6AAAAAA25NXHUE . You are receiving this because you were mentioned.Message ID: @.***>

unicomp21 avatar Aug 01 '23 12:08 unicomp21

We just talked with some folks and they gave us remote control access to their computer and we were testing the extension that way.

ValentaTomas avatar Aug 01 '23 12:08 ValentaTomas

Looking at unified diff in openai evals, looks like we need a different/simpler approach for handling code deltas? I wonder if gpt-4 could supply simple search/replace string? along w/ target filename?

This seems to be the way cursor.so does it

JacobFV avatar Aug 05 '23 22:08 JacobFV