llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

Inference Failed Because of '500 Internal Server Error'

Open dawenxi-007 opened this issue 6 months ago • 5 comments

After launching the distribution server by "llama distribution start --name local-llama-8b --port 5000 --disable-ipv6 ", running any inference example, for example "python examples/scripts/vacation.py localhost 5000 --disable-safety" will give the following error:

Traceback (most recent call last):
  File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/hello.py", line 34, in <module>
    fire.Fire(main)
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/hello.py", line 18, in main                                                                                          [58/582]
    asyncio.run(
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/ubuntu/taoz/llama-agentic-system/examples/scripts/multi_turn.py", line 41, in run_main
    client = await get_agent_system_instance(
  File "/home/ubuntu/taoz/llama-stack/llama_toolchain/agentic_system/utils.py", line 121, in get_agent_system_instance
    create_response = await api.create_agentic_system(create_request)
  File "/home/ubuntu/taoz/llama-stack/llama_toolchain/agentic_system/client.py", line 56, in create_agentic_system
    response.raise_for_status()                                                                                                                                                      [47/582]
  File "/home/ubuntu/miniforge3/envs/llama-stack/lib/python3.10/site-packages/httpx/_models.py", line 761, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://localhost:5000/agentic_system/create'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

dawenxi-007 avatar Aug 28 '24 06:08 dawenxi-007