query.py says build/models not found

[AI-Zebra]

Additional context

/LocalAI/examples/query_data# python query.py /root/miniconda3/lib/python3.11/site-packages/langchain/llms/openai.py:200: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: from langchain.chat_models import ChatOpenAI warnings.warn( /root/miniconda3/lib/python3.11/site-packages/langchain/llms/openai.py:787: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: from langchain.chat_models import ChatOpenAI warnings.warn( Question: which companies gururaj has worked? /embeddings /embeddings /chat/completions Retrying langchain.llms.openai.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised APIError: could not load model - all backends returned error: 25 errors occurred: * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unknown desc = failed loading model * could not load model: rpc error: code = Unavailable desc = error reading from server: EOF * could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory * could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory * could not load model: rpc error: code = Unknown desc = unsupported model type /build/models/gpt-3.5-turbo (should end with .onnx) * backend unsupported: /build/extra/grpc/vall-e-x/ttsvalle.py * backend unsupported: /build/extra/grpc/vllm/backend_vllm.py * backend unsupported: /build/extra/grpc/huggingface/huggingface.py * backend unsupported: /build/extra/grpc/autogptq/autogptq.py * backend unsupported: /build/extra/grpc/bark/ttsbark.py * backend unsupported: /build/extra/grpc/diffusers/backend_diffusers.py * backend unsupported: /build/extra/grpc/exllama/exllama.py

{"error":{"code":500,"message":"could not load model - all backends returned error: 25 errors occurred:\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n\t* could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = unsupported model type /build/models/gpt-3.5-turbo (should end with .onnx)\n\t* backend unsupported: /build/extra/grpc/vall-e-x/ttsvalle.py\n\t* backend unsupported: /build/extra/grpc/vllm/backend_vllm.py\n\t* backend unsupported: /build/extra/grpc/huggingface/huggingface.py\n\t* backend unsupported: /build/extra/grpc/autogptq/autogptq.py\n\t* backend unsupported: /build/extra/grpc/bark/ttsbark.py\n\t* backend unsupported: /build/extra/grpc/diffusers/backend_diffusers.py\n\t* backend unsupported: /build/extra/grpc/exllama/exllama.py\n\n","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: 25 errors occurred:\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n\t* could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = stat /build/models/gpt-3.5-turbo: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = unsupported model type /build/models/gpt-3.5-turbo (should end with .onnx)\n\t* backend unsupported: /build/extra/grpc/vall-e-x/ttsvalle.py\n\t* backend unsupported: /build/extra/grpc/vllm/backend_vllm.py\n\t* backend unsupported: /build/extra/grpc/huggingface/huggingface.py\n\t* backend unsupported: /build/extra/grpc/autogptq/autogptq.py\n\t* backend unsupported: /build/extra/grpc/bark/ttsbark.py\n\t* backend unsupported: /build/extra/grpc/diffusers/backend_diffusers.py\n\t* backend unsupported: /build/extra/grpc/exllama/exllama.py\n\n', 'type': ''}} {'Date': 'Wed, 04 Oct 2023 12:23:15 GMT', 'Content-Type': 'application/json', 'Content-Length': '2188'}.

[Aisuko]

Hi @AI-Zebra . The logs show that localAI cannot load model successfully. And the model does not exist in stat /build/models/gpt-3.5-turbo: no such file or directory.

[Aisuko]

Have you checked if the model already exists in the path models/<your-model-name>

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.