Gemini: parse code execution and grounding result
(I'm not sure if this feature is desired or not. This is a draft version. Please let me know how you think.)
Gemini model support builtin tools including google search (grounding) and code execution. Now users can use :request-params to add support for them (https://github.com/karthink/gptel/issues/750#issuecomment-2815590261 ). This PR extends the parsing function for Gemini to print grounding and code execution results. (We may need better ways other than custom :request-params to support these tools, but that's for future PRs)
@blahgeek Thanks for the PR. This feature is welcome. Can the codeExecution parsing be folded into the main cl-loop? Otherwise this looks fine.
(We may need better ways other than custom
:request-paramsto support these tools, but that's for future PRs)
It's best to expose these features as backend-specific tools, I think. There's no way in gptel or its UI to have tools that only work with one backend, so yeah, :request-params will do for now.
Thanks. I've updated the code according to your suggestion.
@blahgeek Nice! I think it would be interesting to have an option to extract the target redirected URL from the obfuscated "vertexaisearch.cloud.google.com" URLs. In Linux I've been doing this with curl -s -o /dev/null -w '%header{location}' --head <URL>.
Also, a couple of weeks ago I hacked together the insertion of grounding URLs with the associated citations throughout the LLM response (which shows where each grounding applies in the response). Unfortunately gptel's markdown->org conversion makes the :startIndex and :endIndex returned by Gemini incorrect for inserting the citations.
Here's the code, which may help in this PR:
https://gist.github.com/eslgastal/6700418bbad6ff9e49a4d6e28fe3dd83
And a screenshot:
This looks great, are there any blockers to merging this?