LocalAI rpc error

LocalAI version:

LocalAI version: v2.26.0

Environment, CPU architecture, OS, and Version:

2025-03-14 14:46:43 model name : 12th Gen Intel(R) Core(TM) i5-1240P 2025-03-14 14:46:43 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves umip gfni vaes vpclmulqdq rdpid fsrm md_clear flush_l1d arch_capabilities 2025-03-14 14:46:43 CPU: AVX found OK 2025-03-14 14:46:43 CPU: AVX2 found OK 2025-03-14 14:46:43 CPU: no AVX512 found

Describe the bug

didy add the rerank from the localai, localai have error logs:

2025-03-14 15:01:37 7:01AM INF Trying to load the model 'lb-reranker-0.5b-v1.0' with the backend '[llama-cpp llama-cpp-fallback piper silero-vad stablediffusion-ggml whisper bark-cpp huggingface /build/backend/python/diffusers/run.sh /build/backend/python/vllm/run.sh /build/backend/python/autogptq/run.sh /build/backend/python/exllama2/run.sh /build/backend/python/bark/run.sh /build/backend/python/coqui/run.sh /build/backend/python/faster-whisper/run.sh /build/backend/python/kokoro/run.sh /build/backend/python/transformers/run.sh /build/backend/python/rerankers/run.sh]' 2025-03-14 15:01:37 7:01AM INF [llama-cpp] Attempting to load 2025-03-14 15:01:37 7:01AM INF BackendLoader starting backend=llama-cpp modelID=lb-reranker-0.5b-v1.0 o.model=lb-reranker-0.5B-v1.0-Q4_K_M.gguf 2025-03-14 15:01:37 WARNING: failed to determine nodes: open /sys/devices/system/node: no such file or directory 2025-03-14 15:01:37 WARNING: failed to read int from file: open /sys/class/drm/card0/device/numa_node: no such file or directory 2025-03-14 15:01:37 WARNING: failed to determine nodes: open /sys/devices/system/node: no such file or directory 2025-03-14 15:01:37 7:01AM INF [llama-cpp] attempting to load with AVX2 variant 2025-03-14 15:01:47 7:01AM INF Success ip=127.0.0.1 latency="36.725µs" method=GET status=200 url=/readyz 2025-03-14 15:01:47 7:01AM INF [llama-cpp] Loads OK 2025-03-14 15:01:47 7:01AM ERR Server error error="rpc error: code = Unimplemented desc = " ip=172.17.0.1 latency=9.933625673s method=POST status=500 url=/v1/rerank 2025-03-14 15:02:01 7:02AM INF Success ip=172.17.0.1 latency="146.102µs" method=GET status=200 url=/v1/models 2025-03-14 15:02:01 7:02AM INF Success ip=172.17.0.1 latency="22.992µs" method=GET status=200 url=/favicon.ico 2025-03-14 15:02:28 7:02AM ERR Server error error="rpc error: code = Unimplemented desc = " ip=172.17.0.1 latency=3.259999ms method=POST status=500 url=/v1/rerank

2025-03-14 15:05:11 7:05AM INF Success ip=172.17.0.1 latency="130.402µs" method=GET status=200 url=/v1/models 2025-03-14 15:05:11 7:05AM INF Success ip=172.17.0.1 latency="120.759µs" method=GET status=200 url=/favicon.ico 2025-03-14 15:05:16 7:05AM ERR Server error error="rpc error: code = Unimplemented desc = " ip=172.17.0.1 latency=3.314038ms method=POST status=500 url=/v1/rerank

2025-03-14 15:12:25 7:12AM INF BackendLoader starting backend=rerankers modelID=cross-encoder o.model=cross-encoder 2025-03-14 15:12:47 7:12AM INF Success ip=127.0.0.1 latency="28.877µs" method=GET status=200 url=/readyz 2025-03-14 15:13:03 7:13AM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46711: connect: connection refused"" 2025-03-14 15:13:05 7:13AM ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=172.17.0.1 latency=40.049365321s method=POST status=500 url=/v1/rerank

To Reproduce