[Bad Case]: 在机器上部署reranker模型之后，请求的时候报错了

[TOMATODA]

Description / 描述

在机器上部署reranker模型之后，请求的时候报错了 {"error":{"message":"InternalServerError: the loaded moded cannot fullyfill rerank.options are {'embed'} inherited from model_class=<class 'infinity_emb.transformer.embedder.sentence_transformer.SentenceTransformerPatched'>","type":null,"param":null,"code":500}}。 我是从modelscope中获取模型文件的，https://modelscope.cn/models/openbmb/minicpm-reranker

Case Explaination / 案例解释

No response

[Kaguya-19]

请提供您的服务代码

[TOMATODA]

启动命令： command: /app/.venv/bin/python /app/.venv/bin/infinity_emb --model-name-or-path /opt/llm/input/pretrain/OpenBMB--MiniCPM-Reranker --port 8002 --device cuda
imageUrl: pre-openmatrix-acr-registry.cn-beijing.cr.aliyuncs.com/openmatrix/infinity:0.0.32

curl http://localhost:8002/models
{"data":[{"id":"pretrain/OpenBMB--MiniCPM-Reranker","stats":{"queue_fraction":0.0,"queue_absolute":0,"results_pending":0,"batch_size":32},"object":"model","owned_by":"infinity","created":1726209146,"backend":"torch"}],"object":"list"}

不知道提供的信息是否足够，如果还需要其他信息麻烦提醒一下

[Kaguya-19]

启动命令： command: /app/.venv/bin/python /app/.venv/bin/infinity_emb --model-name-or-path /opt/llm/input/pretrain/OpenBMB--MiniCPM-Reranker --port 8002 --device cuda imageUrl: pre-openmatrix-acr-registry.cn-beijing.cr.aliyuncs.com/openmatrix/infinity:0.0.32

curl http://localhost:8002/models {"data":[{"id":"pretrain/OpenBMB--MiniCPM-Reranker","stats":{"queue_fraction":0.0,"queue_absolute":0,"results_pending":0,"batch_size":32},"object":"model","owned_by":"infinity","created":1726209146,"backend":"torch"}],"object":"list"}

不知道提供的信息是否足够，如果还需要其他信息麻烦提醒一下

这边需要您infinity_emb里面的代码，我们初步推测您需要hack进Sentence Transformers库，集成我们的rerank函数https://www.sbert.net/examples/applications/cross-encoder/README.html

[Kaguya-19]

@TOMATODA 我们现已集成对Sentence Transformers库的支持，详见https://huggingface.co/openbmb/MiniCPM-Reranker

[dangfan]

@TOMATODA 请问您解决了这个问题吗？

[Kaguya-19]

您可以尝试将config.jsonl进行修改 "architectures": [   "MiniCPMForSequenceClassification"   ], @dangfan @TOMATODA