llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

return proper error from agent instance to the client

Open chuenlok opened this issue 1 year ago • 2 comments

What does this PR do?

The assert errors are not returned to the client and hid by the 500 internal server error. Append the real error in the error message so the client can show it. Updates on the python client https://github.com/meta-llama/llama-stack-client-python/pull/44

  • [x] Addresses issue (#434)

Test Plan

Please describe:

  • tests you ran to verify your changes with result summaries.
  • provide instructions so it can be reproduced.
  1. Kick off the server with llama stack run /Users/henrytai/.llama/distributions/llamastack-ollama/ollama-run.yaml --port 5050
  2. Trigger an error case from client. The example is PYTHONPATH=. python -m examples.agents.client localhost 5050 from the llama-stack-apps repo. The issue details: "This is a known issue with Llama3.2-3B-Instruct on ollama, where the model produces bad tool calling outputs. In this case, the server agent fails due to assert error that "code_interpreter" is not in the AgentConfig's tools field" but the code_interpreter is not really necessary as it is not needed in Llama3.1-8B.

client log:

inference> assistant<|end_header_id|>

Switzerland is a beautiful country with a rich history, stunning mountains, and vibrant cities. Here are three must-visit places in Switzerland:

1. **Geneva**: Geneva is a global hub for international diplomacy and is often referred to as the "City of Peace". It's home to ...

These are just a few examples of Switzerland's amazing attractions!
500: Internal server error: An unexpected error occurred. Tool code_interpreter not found
inference> brave_search.call(query="what is special about #1")...

Showing the real error in the client: Tool code_interpreter not found

Sources

Please link relevant resources if necessary.

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [X] Ran pre-commit to handle lint / formatting issues.
  • [X] Read the contributor guideline, Pull Request section?
  • [ ] Updated relevant documentation.
  • [ ] Wrote necessary unit or integration tests.

chuenlok avatar Nov 20 '24 20:11 chuenlok