llama-stack
llama-stack copied to clipboard
Inference Failed Because of '500 Internal Server Error'
After launching the distribution server by "llama distribution start --name local-llama-8b --port 5000 --disable-ipv6 "
, running any inference example, for example "python examples/scripts/vacation.py localhost 5000 --disable-safety"
will give the following error:
Traceback (most recent call last):
File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/hello.py", line 34, in <module>
fire.Fire(main)
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/hello.py", line 18, in main [58/582]
asyncio.run(
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/multi_turn.py", line 41, in run_main
client = await get_agent_system_instance(
File "/home/ubuntu/taoz/llama-stack/llama_toolchain/agentic_system/utils.py", line 121, in get_agent_system_instance
create_response = await api.create_agentic_system(create_request)
File "/home/ubuntu/taoz/llama-stack/llama_toolchain/agentic_system/client.py", line 56, in create_agentic_system
response.raise_for_status() [47/582]
File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/httpx/_models.py", line 761, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://localhost:5000/agentic_system/create'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500