Feature request: action that enables AI-driven editing

[wolfmanstout]

Since there are now models that can propose arbitrary edits based on natural language (e.g. GPT-3 flavors), it would be a delight to have this functionality integrated into Cursorless. The simplest option may be to integrate with the upcoming Copilot Chat feature that is now in limited preview. This provides a command which operates on the selected code and brings up a prompt to perform an arbitrary refactoring/edit (see post). This can be currently integrated into actions_custom.csv using copilot, interactiveEditor.start, and then the prompt can be issued with a follow-up command to insert prose and finally press enter. At this point, Copilot will provide a preview of the changes to be applied before they are finally confirmed with control/command-enter. This flow could be slightly improved with a native Cursorless implementation that enables a single command of the form copilot <target> to <prose description of change>, which invokes Copilot, enters the description, and presses enter. The final confirmation can be done separately however the user wishes to press control/command-enter.

[pokey]

Yes was planning to do something like that once I get off the waitlist for copilot chat. Note that you should be able to implement your proposal already using https://www.cursorless.org/docs/user/customization/#example-of-combining-capture-and-action

Eg something like:

copilot <user.cursorless_target> to <phrase>:
    user.cursorless_command("setSelection", cursorless_target)
    user.vscode("interactiveEditor.start")
    sleep(350ms)
    insert(phrase)
    key(enter)

[wolfmanstout]

Works great! I'm using prose instead of phrase so I can leverage custom vocabulary. I also added the following convenience functions that use the "topic" feature of Copilot Chat:

copilot explain <user.cursorless_target> [to <user.prose>]:
    user.cursorless_command("setSelection", cursorless_target)
    user.vscode("interactiveEditor.start")
    sleep(350ms)
    insert("/explain ")
    insert(prose or "")
    key(enter)
copilot test <user.cursorless_target> [to <user.prose>]:
    user.cursorless_command("setSelection", cursorless_target)
    user.vscode("interactiveEditor.start")
    sleep(350ms)
    insert("/test ")
    insert(prose or "")
    key(enter)
copilot fix <user.cursorless_target> [to <user.prose>]:
    user.cursorless_command("setSelection", cursorless_target)
    user.vscode("interactiveEditor.start")
    sleep(350ms)
    insert("/fix ")
    insert(prose or "")
    key(enter)

[pokey]

Nice! You can prob tune that sleep; might be able to get away with something smaller. Or might need to increase it if the chat sometimes takes a bit longer to appear

[wolfmanstout]

I tried removing the sleep altogether, and the failure mode was pretty nasty (replaced selected text) so I kept as is. I imagine it could be further dialed in.

[pokey]

Ok I have some super experimental commands in my personal repo in case anyone wants to play with them. They're in copilot.talon and copilot.py.

There are some commands to interact with the chat sidebar:

• "pilot chat": opens chat to enable typing
• "pilot chat hello world": opens chat, types "hello world" and hits enter
• "pilot bring first": inserts first code block in most recent chat response into your editor at cursor position
• "pilot copy first": copies first code block in most recent chat response
• "pilot bring second last": inserts second-to-last code block in most recent chat response into your editor at cursor position
• "pilot bring first after state air": inserts first code block in most recent chat response into your editor after statement containing air token
• "pilot bring first to funk": replaces function containing your cursor with first code block in most recent chat response

And there are some commands to interact with the inline chat:

• "pilot change funk air" selects function containing air token and opens inline chat to enable typing
• "pilot change funk air to remove argument": selects function containing air token, runs inline chat, types remove argument, and presses enter
• "pilot test funk air": runs inline /test chat on function containing air token, and presses enter
• "pilot fix funk air to remove argument": runs inline chat on function containing air token and types /fix remove argument, and presses enter

I just wired these up now in my personal user files; will see how I get on with them but figured I'd post in case anyone wants to play with them / has ideas for improvements

[pokey]

See also https://github.com/orgs/community/discussions/66365

[pokey]

Ok I made some updates to my experimental Copilot commands to use the newer Copilot refactor quick action, as the inline chat functionality is being deprecated. I also added some docs. See https://github.com/pokey/pokey_talon/tree/main/apps/vscode/copilot