magentic icon indicating copy to clipboard operation
magentic copied to clipboard

Improve handling of exceptions from tool calls in `Chat`

Open jackmpcollins opened this issue 1 year ago • 1 comments

The Chat.exec_function_call should optionally be able to catch exceptions raised when executing the FunctionCall and append an appropriate result message containing details of the error. It should still just make a single function call and append a single message, and any required looping/retry should be handled outside of this method.

Worth considering if this can/should be unified with RetryChatModel? RetryChatModel is concerned with handling errors generating or parsing the LLM response, whereas this would handle exceptions raised by the user-provided functions, so maybe makes sense to keep separate.

Related to https://github.com/jackmpcollins/magentic/issues/417

jackmpcollins avatar Feb 02 '25 22:02 jackmpcollins

  • Cover parameter-validation errors (wrong/missing args) and exceptions that surface mid-stream?
  • Wrap param errors in a new ToolValidationError, and streamed exceptions in ToolStreamError, then convert both to a ToolResultMessage so RetryChatModel can decide to continue/retry ?
  • Add tests: (a) bad params, (b) tool raises, (c) streamed error -> assert chunks are recombined and a single result message is appended.

jackmpcollins avatar Apr 27 '25 01:04 jackmpcollins