[BUG] Got a connection error when I send a request

[lordofthejars]

Expected Behavior

Returns a valid response and not a timeout error

Actual Behavior

Ends up by returning a timeout error. I am unsure if it's a bug or something I am doing wrong, but the example is simple, so maybe I am missing some parameters. The example is simple, so maybe I am missing some parameters.

Running on Docker Engine 20.10.14 on Mac OS.

Steps To Reproduce

#syntax=ghcr.io/sozercan/aikit:latest
apiVersion: v1alpha1
models:
  - name: uncased-sentiment
    source: https://github.com/lordofthejars/bert-base-multilingual-uncased-sentiment_onnx/releases/download/1.0.0/model.onnx 

Then I build:

docker buildx create --use --name aikit-builder

docker buildx build . -t my-model -f aikitfile.yaml --load

Then I start it:

docker run -ti --rm -p 8080:8080 my-model

Finally I curl the model:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "uncased-sentiment",
     "messages": [{"role": "user", "content": "It is so good"}]
   }'

And in the container console I got the following error:

rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:39863: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:39839: connect: connection refused"
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:33679: connect: connection refused"
```

Any idea why this is happening? The example is a simple, not complicated lifecycle.

### Are you willing to submit PRs to contribute to this bug fix?

- [ ] Yes, I am willing to implement it.

[sozercan]

@lordofthejars I am not sure if onnx format is supported by LocalAI yet. You can check the compatibility here: https://localai.io/model-compatibility/

[lordofthejars]

Thank you very much. One question, do you know if any of these files https://huggingface.co/nlptown/bert-base-multilingual-uncased-sentiment/tree/main are supported by LocalAI. I end up loosing a bit between so many formats. Thanks.