gptel-transient: tool selection using completing-read-multiple

[ahmed-shariff]

Building off this comment: https://github.com/karthink/gptel/issues/514#issuecomment-2594947425

This is a proposal for replacing the tool selection sub menu in gptel-menu with completing-read-multiple. This replaces the gptel-tools prefix with infexes to (a) add, (b) remove and (c) remove-all tools. (a) and (b) use completing-read-multiple. The candidate list in both would have the individual tools as well as the categories. When adding, if a category is selected, all tools related to that are added. The same would happen when removing.

A quick demo of what it looks like for now:

https://github.com/user-attachments/assets/1232237e-a10d-44dd-9cdf-77b9b40e4f41

What do you think about this?

TODO:

• [x] annotations

[ahmed-shariff]

Added the annotations:

[psionic-k]

Reviewing the video, seems pretty good. As our tool numbers grow, narrowing and completion seem like the right way. I can easily see the average gptel user having 50 tools by the end of the year.

Did you incidentally create a programming interface so I can set tools by category from an Elisp program?

[ahmed-shariff]

Thanks for taking a look at this.

I had not implemented programming interfaces for that. Though, it should be pretty straightforward to do?

(setq gptel-tools (alist-get "emacs" gptel--known-tools nil nil #'string-equal))

I remember seeing a discussion about having a programmatic interface tool use, but couldn't find it in a rush 😅 If that is of interest, it should be straightforward to add.

[karthink]

Thanks for this PR.

I remember seeing a discussion about having a programmatic interface tool use, but couldn't find it in a rush 😅 If that is of interest, it should be straightforward to add.

gptel-get-tool

---

General comments: The current tools menu definitely does not scale well as the number of tools grows. It's quite easy to end up with 150 tools if you use MCP servers, so some solution is needed. See also: our discussion about the design of the presets menu in #542.

For similar reasons, a separate tools menu is needed: there are other tool-related options that need to be set, and there's no room in gptel-menu. So this PR's version of the completing-read interface won't work.

What would have been ideal is a tools menu with a sub-menu (transient group) that opens to the side of the selected category. For example, the tools menu would look like this by default:

[COMMON OPTIONS]

f filesystem tools
e emacs tools
w web tools

When you press f, the corresponding group shows up on the right:

[COMMON OPTIONS]

f filesystem tools      f (select all)
e emacs tools           r read_file     
w web tools             c create_file   
                        m make_directory
                        l list_directory

So pressing f f selects all filesystem tools, and so on. The keybindings are the same as now, but the display is "2D" and only one category's members are displayed at a time.

Unfortunately I don't know how to get Transient to do this. I can get the 2D layout functional, but when you hide a group, your selections from other groups are lost. For example if you select some filesystem tools above, then open the emacs tools group, the filesystem selections are lost. Transient does not track your selections in hidden/disabled groups.

If anyone has ideas on how to get Transient to do this, I'm all ears.

---

The next best alternative would be to have the categories as is, but allow selecting individual tools for a category using completing-read-multiple:

[COMMON OPTIONS]

f s filesystem tools (0/4)   f f select all
e s emacs tools (0/5)        e e select all
w s web tools (0/4)          w s select all

Pressing f s opens up a completing-read-multiple interface, and pressing f f selects all filesystem tools immediately. (A better keybinding scheme can be found.)

[ahmed-shariff]

a separate tools menu is needed: there are other tool-related options that need to be set

Can this PR's interface be part of the sub-menu?

the tools menu would look like this by default:

[COMMON OPTIONS]

f filesystem tools
e emacs tools
w web tools

When you press f, the corresponding group shows up on the right:

[COMMON OPTIONS]

f filesystem tools      f (select all)
e emacs tools           r read_file     
w web tools             c create_file   
                        m make_directory
                        l list_directory

The next best alternative would be to have the categories as is, but allow selecting individual tools for a category using completing-read-multiple:

[COMMON OPTIONS]

f s filesystem tools (0/4)   f f select all
e s emacs tools (0/5)        e e select all
w s web tools (0/4)          w s select all

Would these not have the same scaling problem? Also, if I am configuring multiple specific tools from different categories, wouldn't this involve more steps than necessary? I can certainly take a crack at the above design, but I am not clear why the completing-read-multiple interface would not be preferable here.

[karthink]

a separate tools menu is needed: there are other tool-related options that need to be set

Can this PR's interface be part of the sub-menu?

Yes, I think that will work!

The next best alternative would be to have the categories as is, but allow selecting individual tools for a category using completing-read-multiple:

Would these not have the same scaling problem?

No. After using a large number of tools (thanks to MCP) for a week, I realized a few things:

1. Most users won't have more than 15-20 tool categories, and that's on the high side. 10 is more typical.
2. Most of the time, you just pick a category or two and go. (By which I mean you pick all tools in a category.)
3. If you do want to pick individual tools, you usually pick only one or two at a time.

15-20 categories can easily be accommodated in the transient menu.

Also, if I am configuring multiple specific tools from different categories, wouldn't this involve more steps than necessary? I can certainly take a crack at the above design, but I am not clear why the completing-read-multiple interface would not be preferable here.

Selecting multiple tools across categories is quite rare. It's more common to want to

• pick multiple (but not all) tools from one category, or
• pick a couple of categories quickly.

I've actually only had to select multiple tools across categories once, when I wanted the LLM to search online for something (web category) and write a report about it to a file (filesystem category). And even in this case, selecting the web and filesystem categories instead of individual tools would have been faster, and it would have worked fine.

Having explicit keys per category -- one key to select all and one to start completing-read-multiple for that category -- makes this convenient.

Another advantage over a single completing-read-multiple is that you can see all the categories at once in the menu, instead of them being mixed with individual tools because of completing-read's sorting/history method.

[ahmed-shariff]

I see your point about being able to see the items quickly. Let me try a few things and see what I can get to work.

[ahmed-shariff]

Here's an updated version of the transient menu with also a widget showing the selected tools and the changes being made.

https://github.com/user-attachments/assets/f75d79fd-2001-4a0f-accc-ca5f246aff85

[ahmed-shariff]

the tools menu would look like this by default:

[COMMON OPTIONS]

f filesystem tools
e emacs tools
w web tools

When you press f, the corresponding group shows up on the right:

[COMMON OPTIONS]

f filesystem tools      f (select all)
e emacs tools           r read_file     
w web tools             c create_file   
                        m make_directory
                        l list_directory

The next best alternative would be to have the categories as is, but allow selecting individual tools for a category using completing-read-multiple:

[COMMON OPTIONS]

f s filesystem tools (0/4)   f f select all
e s emacs tools (0/5)        e e select all
w s web tools (0/4)          w s select all

Sooo, I took a stab at this:

https://github.com/user-attachments/assets/92fce400-2148-4c91-a20a-87e6e2bace18

I didn't merge this here, its on https://github.com/ahmed-shariff/gptel/tree/transient-tool-alt-menu if you want to try it out. Let me know what you guys think of the two options.

[karthink]

Sooo, I took a stab at this:

https://github.com/user-attachments/assets/92fce400-2148-4c91-a20a-87e6e2bace18

Wow that looks very promising! You're a Transient wizard.

I'll look into how you managed this when I next work on the tools menu again.

[karthink]

@ahmed-shariff Inspired by your demo, I've redesigned gptel's tools menu to use a dynamic two-column layout: 6d42fd25915ffdecc8f2490bf1bb586a492417c3

It should scale better to hundreds of tools (as tends to happen with MCP servers), and also requires fewer keystrokes for selecting/toggling tools or categories.

The main differences from your design are that (i) it does not use global elisp variables for storing intermediate state, and (ii) it shows less information (while hopefully still showing enough).

[ahmed-shariff]

Nice! The new menu looks good. Can we also add an option to clear all tools?

[karthink]

Nice! The new menu looks good. Can we also add an option to clear all tools?

Yes, this is required -- there is a version in #814 but it's out of date now.

[karthink]

I don't plan to add completing-read-multiple based tool selection now as I think the current solution is faster/better at the current scale of up to ~100 tools.

However, there is nothing wrong with the idea. It's an alternative we can consider in the future if the average number of gptel tools grows by an order of magnitude, to 500+.

Can I close this PR now?

[ahmed-shariff]

Hey, yeah, we can close this PR for now. If needed, we can revisit this.