inference
inference copied to clipboard
Published
1 month ago
•
xorbitsai
xorbitsai
模型指定下载中心仍无法拉取成功
明明指定下载中心是modelscope, 为什么还是会去huggingface下载
Server error: 500 - [address=0.0.0.0:46363, pid=1579] We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
简单来说就是这个模型当中的一些设置动态依赖了 huggingface，会依然去 hf 下载这些东西。
我们优化下吧，晚点我们会自己传一个模型到 modelscope 把依赖 hf 的部分替换掉。
试了下，这个模型用了太多动态加载，不好处理。
还是设置 HF 环境变量吧。
HF_ENDPOINT=https://hf-mirror.com
请问是在容器内部设置代理吗，我这样子配置还是不行🤔
This issue is stale because it has been open for 7 days with no activity.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue was closed because it has been inactive for 5 days since being marked as stale.
我从0 开始加载这个模型，没有问题，调用也正常。
In [1]: from xinference.client import Client
In [2]: client = Client('http://gpu:36666')
In [3]: model = client.get_model('jina-embeddings-v3')
In [4]: model.create_embedding("What is the capital of China?")
Out[4]:
{'object': 'list',
'model': 'jina-embeddings-v3',
'model_replica': 'jina-embeddings-v3-0',
'data': [{'index': 0,
'object': 'embedding',
'embedding': [0.052001953125,
-0.09423828125,
-0.005218505859375,
0.048828125,
-0.0322265625,
-0.0205078125,
-0.0693359375,
0.0400390625,
-0.10693359375,
0.0257568359375,
-0.0224609375,
0.0986328125,
-0.005401611328125,
0.05615234375,
-0.1435546875,
-0.15625,
-0.046142578125,
-0.0247802734375,
-0.0135498046875,
-0.021484375,
0.00311279296875,
0.007232666015625,
-0.0302734375,
0.07958984375,
-0.019775390625,
0.003082275390625,
0.08642578125,
0.134765625,
0.0216064453125,
0.049560546875,
0.08984375,
-0.05908203125,
0.09619140625,
-0.06396484375,
-0.02099609375,
-0.006591796875,
-0.0211181640625,
-0.09912109375,
-0.0478515625,
-0.0123291015625,
0.0390625,
0.0220947265625,
0.0022125244140625,
0.01531982421875,
0.04052734375,
0.03857421875,
-0.032470703125,
-0.044921875,
-0.029052734375,
-0.02197265625,
0.0771484375,
0.039794921875,
-0.08642578125,
-0.023193359375,
0.06982421875,
-0.1171875,
-0.0673828125,
-0.0703125,
0.0322265625,
-0.06103515625,
-0.0546875,
-0.021240234375,
0.0257568359375,
-0.064453125,
-0.05810546875,
-0.047119140625,
0.01092529296875,
-0.02490234375,
-0.03271484375,
-0.0238037109375,
-0.053955078125,
-0.029052734375,
0.0498046875,
-0.0263671875,
-0.00811767578125,
-0.0164794921875,
0.0191650390625,
-0.025390625,
-0.056884765625,
-0.03369140625,
0.1259765625,
-0.02392578125,
-0.0146484375,
0.01422119140625,
-0.057861328125,
-0.048828125,
-0.00165557861328125,
0.07177734375,
-0.0242919921875,
0.07470703125,
-0.021728515625,
-0.01123046875,
-0.004638671875,
-0.01019287109375,
0.0771484375,
-0.007568359375,
0.007171630859375,
-0.02490234375,
0.0093994140625,
0.035888671875,
0.025390625,
0.034912109375,
-0.0164794921875,
-0.00775146484375,
0.004547119140625,
0.044677734375,
0.042236328125,
-0.00726318359375,
-0.1162109375,
0.07080078125,
0.006591796875,
0.07080078125,
-0.0157470703125,
0.059326171875,
-0.036865234375,
0.029296875,
0.06005859375,
0.0341796875,
0.01287841796875,
0.020751953125,
-0.021484375,
0.040283203125,
-0.005615234375,
-0.006683349609375,
-0.04248046875,
-0.0198974609375,
-0.05615234375,
-0.043212890625,
0.0198974609375,
0.02197265625,
-0.0152587890625,
0.040771484375,
-0.05078125,
-0.03271484375,
-0.006866455078125,
0.0283203125,
0.05810546875,
0.0712890625,
-0.046875,
0.0267333984375,
0.043212890625,
0.021484375,
-0.011962890625,
0.03466796875,
-0.0003795623779296875,
0.00616455078125,
-0.042724609375,
-0.046875,
-0.04248046875,
0.0130615234375,
0.07470703125,
-0.060546875,
0.00750732421875,
-0.051025390625,
-0.01171875,
-0.02294921875,
-0.0011444091796875,
0.00921630859375,
0.05810546875,
-0.0654296875,
0.030517578125,
0.0257568359375,
-0.042724609375,
0.01434326171875,
0.037841796875,
0.0135498046875,
-0.0068359375,
0.010986328125,
0.004150390625,
0.0113525390625,
-0.01171875,
0.006866455078125,
0.0274658203125,
-0.0299072265625,
0.01336669921875,
-0.0113525390625,
0.041015625,
0.0177001953125,
-0.0693359375,
0.01611328125,
-0.048583984375,
0.0299072265625,
0.03564453125,
0.05224609375,
-0.0186767578125,
0.0247802734375,
-0.021240234375,
0.08251953125,
-0.0186767578125,
0.057373046875,
-0.01214599609375,
-0.007659912109375,
-0.0152587890625,
-0.01385498046875,
0.027099609375,
0.0234375,
0.00274658203125,
-0.01434326171875,
-0.01422119140625,
-0.03271484375,
-0.01483154296875,
-0.041259765625,
0.04931640625,
-0.013671875,
-0.016845703125,
0.036865234375,
-0.052978515625,
0.039794921875,
-0.041259765625,
-0.044921875,
0.0068359375,
-0.0230712890625,
-0.02294921875,
0.031005859375,
0.0089111328125,
-0.00738525390625,
-0.00750732421875,
0.0194091796875,
0.0260009765625,
0.022216796875,
0.032958984375,
0.00927734375,
-0.0269775390625,
0.0267333984375,
-0.0152587890625,
0.031005859375,
0.05859375,
0.0194091796875,
0.0146484375,
-0.049560546875,
0.051513671875,
0.03955078125,
-0.00616455078125,
-0.00726318359375,
0.0211181640625,
-0.000522613525390625,
-0.019287109375,
0.00121307373046875,
-0.026123046875,
0.07373046875,
0.0020751953125,
0.0023651123046875,
-0.05810546875,
-0.044921875,
-0.07470703125,
-0.0079345703125,
0.01397705078125,
-0.009033203125,
0.041748046875,
-0.031982421875,
-0.06005859375,
-0.038818359375,
0.024658203125,
0.03564453125,
0.010498046875,
-0.0152587890625,
0.0150146484375,
-0.021484375,
0.04638671875,
0.003173828125,
-0.0037384033203125,
0.00225830078125,
0.01953125,
0.002838134765625,
0.0014495849609375,
0.03369140625,
0.02734375,
-0.0145263671875,
-0.01153564453125,
-0.025146484375,
-0.01806640625,
0.0260009765625,
-0.01416015625,
-0.004852294921875,
-0.004638671875,
-1.800060272216797e-05,
-0.0302734375,
0.03857421875,
-0.0177001953125,
0.006591796875,
0.0269775390625,
0.01324462890625,
0.0078125,
-0.016845703125,
-0.01318359375,
0.00421142578125,
-0.016357421875,
0.051513671875,
0.0234375,
0.0152587890625,
-0.0155029296875,
0.0184326171875,
0.022705078125,
0.0238037109375,
-0.006439208984375,
0.047119140625,
-0.0035400390625,
0.00092315673828125,
-0.006683349609375,
-0.00421142578125,
-0.00860595703125,
0.05908203125,
0.0050048828125,
-0.053466796875,
0.048095703125,
0.00830078125,
-0.043701171875,
-0.02294921875,
-0.011962890625,
-0.041748046875,
0.033447265625,
-0.018798828125,
-0.0308837890625,
-0.0079345703125,
-0.041748046875,
-0.024658203125,
-0.0277099609375,
0.0050048828125,
-0.006011962890625,
0.041259765625,
0.00567626953125,
0.0029754638671875,
-0.00018787384033203125,
-0.00421142578125,
-0.005523681640625,
-0.0458984375,
-0.011962890625,
0.059814453125,
-0.0478515625,
-0.030029296875,
0.0108642578125,
-0.00958251953125,
0.00567626953125,
0.03662109375,
0.05615234375,
-0.005615234375,
0.01123046875,
0.02392578125,
-0.0244140625,
-0.0029754638671875,
0.05419921875,
-0.0059814453125,
0.0103759765625,
0.004547119140625,
-0.01422119140625,
0.041748046875,
0.0751953125,
0.0106201171875,
0.01373291015625,
-0.03466796875,
-0.006866455078125,
0.06103515625,
-0.01611328125,
-0.0257568359375,
-0.0230712890625,
0.04150390625,
-0.0145263671875,
0.03564453125,
0.03466796875,
-0.0018310546875,
0.055419921875,
5.078315734863281e-05,
-0.041748046875,
0.00274658203125,
0.00177764892578125,
-0.03857421875,
-0.003662109375,
0.012451171875,
-0.016845703125,
-0.01708984375,
0.0198974609375,
-0.040771484375,
-0.010986328125,
-0.01446533203125,
-0.01300048828125,
0.007659912109375,
0.0113525390625,
0.1201171875,
0.000736236572265625,
-0.001373291015625,
0.034423828125,
-0.0235595703125,
-0.0019073486328125,
0.0247802734375,
-0.00177001953125,
0.031005859375,
0.044921875,
-0.022216796875,
0.006744384765625,
-0.052734375,
0.05615234375,
-0.0264892578125,
-0.042724609375,
0.0045166015625,
-0.0185546875,
-0.00616455078125,
0.046142578125,
0.0289306640625,
0.015869140625,
0.0081787109375,
0.018798828125,
-0.0036163330078125,
0.005950927734375,
0.004547119140625,
-0.039306640625,
0.0030059814453125,
-0.037353515625,
-0.02294921875,
0.022705078125,
-0.0068359375,
-0.03857421875,
0.019287109375,
0.0341796875,
0.0264892578125,
0.01611328125,
0.035888671875,
-0.022705078125,
0.042236328125,
-0.006744384765625,
-0.0111083984375,
-0.000946044921875,
-0.029052734375,
-0.05615234375,
-0.0208740234375,
0.00885009765625,
0.036865234375,
-0.036865234375,
-0.052001953125,
0.04150390625,
-0.05029296875,
0.026123046875,
-0.054931640625,
-0.039794921875,
-0.0224609375,
0.043212890625,
0.0019989013671875,
0.00897216796875,
-0.04638671875,
0.00066375732421875,
0.0306396484375,
-0.03271484375,
0.053955078125,
-0.048095703125,
-0.0244140625,
-0.044921875,
-0.046142578125,
-0.06982421875,
0.0279541015625,
-0.0267333984375,
-0.0238037109375,
-0.0274658203125,
0.0242919921875,
0.007080078125,
-0.0306396484375,
-0.007659912109375,
-0.04931640625,
0.02099609375,
-0.02490234375,
0.0240478515625,
0.0157470703125,
-0.0005035400390625,
-0.007171630859375,
0.042236328125,
0.034912109375,
0.052001953125,
0.0185546875,
0.00738525390625,
-0.034423828125,
0.03369140625,
-0.05029296875,
0.01043701171875,
-0.0145263671875,
0.005035400390625,
0.0654296875,
-0.00811767578125,
-0.056884765625,
-0.03564453125,
0.0157470703125,
-0.01190185546875,
-0.025146484375,
-0.01239013671875,
0.0184326171875,
0.0498046875,
0.03173828125,
-0.024658203125,
-0.008056640625,
0.040771484375,
0.01220703125,
0.0458984375,
0.0257568359375,
0.0673828125,
-0.048828125,
0.01239013671875,
0.029296875,
0.007568359375,
0.00946044921875,
0.005615234375,
0.01214599609375,
-0.016845703125,
0.037841796875,
-0.03173828125,
-0.0286865234375,
0.03564453125,
-0.00193023681640625,
0.0155029296875,
0.025146484375,
0.00897216796875,
0.003692626953125,
-0.0306396484375,
0.0206298828125,
0.035400390625,
-0.029052734375,
-0.0244140625,
-0.0140380859375,
-0.0101318359375,
-0.01123046875,
-0.004486083984375,
-0.041259765625,
-0.0224609375,
-0.0284423828125,
-0.028076171875,
-0.0146484375,
0.03564453125,
-0.059814453125,
-0.038330078125,
0.01123046875,
0.01043701171875,
-0.0269775390625,
0.000885009765625,
-0.00848388671875,
-0.0079345703125,
-0.035400390625,
-0.028076171875,
-0.01202392578125,
0.005523681640625,
0.03515625,
-0.0123291015625,
-0.0267333984375,
-0.00994873046875,
0.0186767578125,
0.06103515625,
-0.00168609619140625,
-0.0164794921875,
-0.032470703125,
0.004364013671875,
0.031005859375,
-0.031982421875,
-0.052734375,
0.0186767578125,
0.00159454345703125,
-0.01806640625,
-0.0011138916015625,
-0.0267333984375,
-0.01806640625,
0.04296875,
-0.0181884765625,
-0.005615234375,
-0.0181884765625,
0.00439453125,
0.024658203125,
0.01153564453125,
0.00555419921875,
-0.00341796875,
-0.006072998046875,
0.01373291015625,
-0.029541015625,
0.01007080078125,
-0.0296630859375,
0.0177001953125,
0.0019073486328125,
0.017333984375,
-0.046142578125,
-0.005096435546875,
-0.0260009765625,
0.0111083984375,
-0.0242919921875,
0.010498046875,
0.030029296875,
-0.03662109375,
0.0198974609375,
0.0439453125,
0.01275634765625,
-0.0224609375,
-0.03466796875,
-0.019287109375,
0.0184326171875,
-0.0306396484375,
-0.01220703125,
-0.041748046875,
-0.0177001953125,
-0.01080322265625,
0.037841796875,
0.011962890625,
-0.0302734375,
0.021240234375,
0.01275634765625,
-0.0223388671875,
-0.033203125,
-0.0033416748046875,
-0.035400390625,
-0.03271484375,
-0.0242919921875,
-0.0081787109375,
0.04248046875,
0.0169677734375,
0.0296630859375,
0.048828125,
0.0194091796875,
0.0230712890625,
-0.0045166015625,
0.004852294921875,
0.0203857421875,
0.0205078125,
0.0026702880859375,
-0.0038299560546875,
0.01239013671875,
0.0001068115234375,
-0.0208740234375,
0.0093994140625,
-0.00860595703125,
-0.07080078125,
-0.0146484375,
0.0194091796875,
0.01007080078125,
0.009521484375,
-0.0189208984375,
0.029541015625,
0.02490234375,
-0.0216064453125,
0.00078582763671875,
0.02294921875,
0.0390625,
-0.0177001953125,
-0.06005859375,
-0.005157470703125,
-0.01287841796875,
-0.046142578125,
-0.0177001953125,
0.029296875,
-0.00732421875,
-0.01300048828125,
0.0079345703125,
-0.00921630859375,
-0.0145263671875,
0.008056640625,
-0.00244140625,
-0.00165557861328125,
-0.01043701171875,
0.0032501220703125,
-0.0009918212890625,
-0.044677734375,
0.05908203125,
-0.00897216796875,
-0.036865234375,
-0.006591796875,
-0.038818359375,
-0.0167236328125,
0.0244140625,
-0.0146484375,
-0.01611328125,
-0.037841796875,
0.03759765625,
0.04443359375,
0.01953125,
0.00921630859375,
-0.006439208984375,
-0.00078582763671875,
-0.0234375,
0.01275634765625,
0.0308837890625,
-0.0145263671875,
0.0019073486328125,
0.03955078125,
-0.0186767578125,
0.0255126953125,
0.007659912109375,
0.017578125,
0.0068359375,
0.000278472900390625,
0.0279541015625,
-0.01318359375,
0.011962890625,
0.041015625,
0.027099609375,
0.0242919921875,
-0.01153564453125,
0.01019287109375,
-0.03564453125,
-0.01434326171875,
-0.039794921875,
0.00341796875,
-0.0130615234375,
0.0081787109375,
0.0264892578125,
0.0230712890625,
-0.0079345703125,
0.0279541015625,
-0.0247802734375,
0.00787353515625,
0.052001953125,
0.0086669921875,
0.00787353515625,
-0.000885009765625,
-0.00162506103515625,
0.01287841796875,
0.021728515625,
-0.008056640625,
0.00067138671875,
0.0260009765625,
-0.01287841796875,
0.002685546875,
-0.032470703125,
-0.0157470703125,
-0.004302978515625,
0.00010585784912109375,
-0.0174560546875,
0.0267333984375,
-0.01068115234375,
0.01068115234375,
-0.018310546875,
-0.0498046875,
0.042724609375,
-0.0184326171875,
-0.0028839111328125,
-0.0306396484375,
-0.0361328125,
0.0216064453125,
-0.002105712890625,
-0.030517578125,
-0.0306396484375,
0.03369140625,
-0.0172119140625,
0.0037384033203125,
-0.022705078125,
-0.01287841796875,
0.0238037109375,
0.03369140625,
0.031494140625,
-0.0084228515625,
0.02392578125,
0.01318359375,
0.0269775390625,
0.018310546875,
0.01123046875,
0.0230712890625,
-0.0390625,
-0.017822265625,
0.00067138671875,
0.033935546875,
0.015869140625,
0.0033111572265625,
-0.01177978515625,
-0.0096435546875,
0.0244140625,
0.0054931640625,
0.01123046875,
-0.0302734375,
5.4836273193359375e-06,
0.039794921875,
-0.0184326171875,
0.028076171875,
0.0546875,
-0.03173828125,
-0.000823974609375,
-0.0022430419921875,
-0.0167236328125,
-0.0546875,
0.009765625,
-0.037841796875,
0.01019287109375,
0.005157470703125,
-0.01019287109375,
0.023193359375,
0.0213623046875,
-0.00537109375,
0.00109100341796875,
-0.00946044921875,
-0.0380859375,
0.0264892578125,
-0.0024566650390625,
-0.0279541015625,
0.0035552978515625,
-0.022216796875,
0.03369140625,
-0.0087890625,
-0.010498046875,
0.006072998046875,
0.00970458984375,
0.006866455078125,
-0.00174713134765625,
0.0277099609375,
0.0213623046875,
0.043701171875,
-0.00146484375,
0.037109375,
-0.0164794921875,
-0.0001983642578125,
0.01080322265625,
-0.007568359375,
-0.024658203125,
-0.00262451171875,
-0.007476806640625,
0.0150146484375,
-0.0115966796875,
0.0322265625,
0.00179290771484375,
0.0172119140625,
-0.0220947265625,
-0.02001953125,
-0.036865234375,
0.052734375,
-0.007568359375,
-0.01019287109375,
-0.021240234375,
-0.003143310546875,
-0.00946044921875,
-0.0037078857421875,
-0.0038299560546875,
-0.0191650390625,
0.00506591796875,
-0.0052490234375,
-0.0299072265625,
0.012451171875,
-0.033447265625,
0.03173828125,
-0.01434326171875,
0.01080322265625,
0.00872802734375,
-0.02490234375,
0.0086669921875,
-0.01611328125,
-0.01068115234375,
0.0159912109375,
-0.0150146484375,
0.0150146484375,
0.021484375,
-0.044189453125,
-0.01385498046875,
0.0113525390625,
-0.01068115234375,
-0.0172119140625,
-0.00051116943359375,
0.01904296875,
-0.016357421875,
-0.0152587890625,
0.0184326171875,
0.00811767578125,
-0.0185546875,
-0.0084228515625,
-0.0361328125,
-0.0238037109375,
-0.01275634765625,
0.022705078125,
0.003143310546875,
0.00738525390625,
-0.02734375,
0.00274658203125,
0.01104736328125,
-0.002655029296875,
-0.0264892578125,
-0.040771484375,
0.00396728515625,
0.0026702880859375,
-0.00921630859375,
0.000568389892578125,
0.00396728515625,
0.00970458984375,
0.0322265625,
0.00421142578125,
-0.046142578125,
0.032470703125,
-0.00023651123046875,
0.003265380859375,
0.02392578125,
-0.029296875,
0.01348876953125,
0.016845703125,
-0.023193359375,
0.003692626953125,
0.043212890625,
-0.0027923583984375,
0.017333984375,
0.00897216796875,
-0.0084228515625,
-0.019287109375,
-0.00060272216796875,
0.0296630859375,
0.01055908203125,
-0.01483154296875,
0.00634765625,
-0.0181884765625,
4.649162292480469e-05,
-0.000675201416015625,
0.033447265625,
-0.046142578125,
-0.0279541015625,
-0.00127410888671875,
-0.0242919921875,
0.04638671875,
0.0023040771484375,
-0.046142578125,
0.06640625,
0.0279541015625,
0.01324462890625,
0.017822265625,
-0.02001953125,
0.01190185546875,
-0.008544921875,
-0.0024871826171875,
-0.0186767578125,
0.027099609375,
9.1552734375e-05,
-0.00091552734375,
-0.006683349609375,
0.017333984375,
0.0101318359375,
-0.01300048828125,
-0.013671875,
-0.0115966796875,
0.006591796875,
-0.0162353515625,
-0.0263671875,
0.00994873046875,
-0.04931640625,
0.0289306640625,
0.039794921875,
-0.01416015625,
-0.026123046875,
0.03564453125,
-0.024658203125,
0.01544189453125,
-0.006927490234375,
0.01531982421875,
0.002197265625,
0.01025390625,
-0.0230712890625,
0.01092529296875,
0.01177978515625,
0.025390625,
-0.003631591796875,
-0.0186767578125,
0.021484375,
-0.055908203125,
0.03369140625,
0.02734375,
-0.02197265625,
0.0191650390625,
0.022705078125,
0.0023040771484375,
-0.0257568359375,
0.0103759765625,
0.0255126953125,
-0.0034637451171875,
0.003662109375,
0.0140380859375,
-0.0084228515625,
0.054931640625,
0.03662109375,
-0.01177978515625,
0.02392578125,
0.00616455078125,
0.0135498046875,
0.01336669921875,
0.01422119140625,
0.000812530517578125,
-0.03271484375,
0.01043701171875,
0.006500244140625,
0.0184326171875,
-0.00177001953125,
-0.0005950927734375,
0.00811767578125,
0.00946044921875,
0.0274658203125,
0.00131988525390625,
-0.0003795623779296875,
-0.01324462890625,
0.0216064453125,
0.0084228515625,
0.054931640625,
-0.021240234375,
0.0242919921875,
-0.0086669921875,
0.05224609375,
-0.01007080078125,
0.046875,
0.00738525390625,
0.057373046875,
0.0260009765625,
-0.0174560546875,
0.0123291015625,
-0.01446533203125,
0.00616455078125,
-0.00238037109375,
0.0264892578125,
-0.0027313232421875,
0.0166015625,
-0.00970458984375,
0.0096435546875,
0.01287841796875,
...]}],
'usage': {'prompt_tokens': 9, 'total_tokens': 9}}
启动 xinference 设置环境变量：
XINFERENCE_MODEL_SRC=modelscope HF_ENDPOINT=https://hf-mirror.com xinference-local
我从0 开始加载这个模型，没有问题，调用也正常。
In [1]: from xinference.client import Client In [2]: client = Client('http://gpu:36666') In [3]: model = client.get_model('jina-embeddings-v3') In [4]: model.create_embedding("What is the capital of China?") Out[4]: {'object': 'list', 'model': 'jina-embeddings-v3', 'model_replica': 'jina-embeddings-v3-0', 'data': [{'index': 0, 'object': 'embedding', 'embedding': [0.052001953125, -0.09423828125, -0.005218505859375, 0.048828125, -0.0322265625, -0.0205078125, -0.0693359375, 0.0400390625, -0.10693359375, 0.0257568359375, -0.0224609375, 0.0986328125, -0.005401611328125, 0.05615234375, -0.1435546875, -0.15625, -0.046142578125, -0.0247802734375, -0.0135498046875, -0.021484375, 0.00311279296875, 0.007232666015625, -0.0302734375, 0.07958984375, -0.019775390625, 0.003082275390625, 0.08642578125, 0.134765625, 0.0216064453125, 0.049560546875, 0.08984375, -0.05908203125, 0.09619140625, -0.06396484375, -0.02099609375, -0.006591796875, -0.0211181640625, -0.09912109375, -0.0478515625, -0.0123291015625, 0.0390625, 0.0220947265625, 0.0022125244140625, 0.01531982421875, 0.04052734375, 0.03857421875, -0.032470703125, -0.044921875, -0.029052734375, -0.02197265625, 0.0771484375, 0.039794921875, -0.08642578125, -0.023193359375, 0.06982421875, -0.1171875, -0.0673828125, -0.0703125, 0.0322265625, -0.06103515625, -0.0546875, -0.021240234375, 0.0257568359375, -0.064453125, -0.05810546875, -0.047119140625, 0.01092529296875, -0.02490234375, -0.03271484375, -0.0238037109375, -0.053955078125, -0.029052734375, 0.0498046875, -0.0263671875, -0.00811767578125, -0.0164794921875, 0.0191650390625, -0.025390625, -0.056884765625, -0.03369140625, 0.1259765625, -0.02392578125, -0.0146484375, 0.01422119140625, -0.057861328125, -0.048828125, -0.00165557861328125, 0.07177734375, -0.0242919921875, 0.07470703125, -0.021728515625, -0.01123046875, -0.004638671875, -0.01019287109375, 0.0771484375, -0.007568359375, 0.007171630859375, -0.02490234375, 0.0093994140625, 0.035888671875, 0.025390625, 0.034912109375, -0.0164794921875, -0.00775146484375, 0.004547119140625, 0.044677734375, 0.042236328125, -0.00726318359375, -0.1162109375, 0.07080078125, 0.006591796875, 0.07080078125, -0.0157470703125, 0.059326171875, -0.036865234375, 0.029296875, 0.06005859375, 0.0341796875, 0.01287841796875, 0.020751953125, -0.021484375, 0.040283203125, -0.005615234375, -0.006683349609375, -0.04248046875, -0.0198974609375, -0.05615234375, -0.043212890625, 0.0198974609375, 0.02197265625, -0.0152587890625, 0.040771484375, -0.05078125, -0.03271484375, -0.006866455078125, 0.0283203125, 0.05810546875, 0.0712890625, -0.046875, 0.0267333984375, 0.043212890625, 0.021484375, -0.011962890625, 0.03466796875, -0.0003795623779296875, 0.00616455078125, -0.042724609375, -0.046875, -0.04248046875, 0.0130615234375, 0.07470703125, -0.060546875, 0.00750732421875, -0.051025390625, -0.01171875, -0.02294921875, -0.0011444091796875, 0.00921630859375, 0.05810546875, -0.0654296875, 0.030517578125, 0.0257568359375, -0.042724609375, 0.01434326171875, 0.037841796875, 0.0135498046875, -0.0068359375, 0.010986328125, 0.004150390625, 0.0113525390625, -0.01171875, 0.006866455078125, 0.0274658203125, -0.0299072265625, 0.01336669921875, -0.0113525390625, 0.041015625, 0.0177001953125, -0.0693359375, 0.01611328125, -0.048583984375, 0.0299072265625, 0.03564453125, 0.05224609375, -0.0186767578125, 0.0247802734375, -0.021240234375, 0.08251953125, -0.0186767578125, 0.057373046875, -0.01214599609375, -0.007659912109375, -0.0152587890625, -0.01385498046875, 0.027099609375, 0.0234375, 0.00274658203125, -0.01434326171875, -0.01422119140625, -0.03271484375, -0.01483154296875, -0.041259765625, 0.04931640625, -0.013671875, -0.016845703125, 0.036865234375, -0.052978515625, 0.039794921875, -0.041259765625, -0.044921875, 0.0068359375, -0.0230712890625, -0.02294921875, 0.031005859375, 0.0089111328125, -0.00738525390625, -0.00750732421875, 0.0194091796875, 0.0260009765625, 0.022216796875, 0.032958984375, 0.00927734375, -0.0269775390625, 0.0267333984375, -0.0152587890625, 0.031005859375, 0.05859375, 0.0194091796875, 0.0146484375, -0.049560546875, 0.051513671875, 0.03955078125, -0.00616455078125, -0.00726318359375, 0.0211181640625, -0.000522613525390625, -0.019287109375, 0.00121307373046875, -0.026123046875, 0.07373046875, 0.0020751953125, 0.0023651123046875, -0.05810546875, -0.044921875, -0.07470703125, -0.0079345703125, 0.01397705078125, -0.009033203125, 0.041748046875, -0.031982421875, -0.06005859375, -0.038818359375, 0.024658203125, 0.03564453125, 0.010498046875, -0.0152587890625, 0.0150146484375, -0.021484375, 0.04638671875, 0.003173828125, -0.0037384033203125, 0.00225830078125, 0.01953125, 0.002838134765625, 0.0014495849609375, 0.03369140625, 0.02734375, -0.0145263671875, -0.01153564453125, -0.025146484375, -0.01806640625, 0.0260009765625, -0.01416015625, -0.004852294921875, -0.004638671875, -1.800060272216797e-05, -0.0302734375, 0.03857421875, -0.0177001953125, 0.006591796875, 0.0269775390625, 0.01324462890625, 0.0078125, -0.016845703125, -0.01318359375, 0.00421142578125, -0.016357421875, 0.051513671875, 0.0234375, 0.0152587890625, -0.0155029296875, 0.0184326171875, 0.022705078125, 0.0238037109375, -0.006439208984375, 0.047119140625, -0.0035400390625, 0.00092315673828125, -0.006683349609375, -0.00421142578125, -0.00860595703125, 0.05908203125, 0.0050048828125, -0.053466796875, 0.048095703125, 0.00830078125, -0.043701171875, -0.02294921875, -0.011962890625, -0.041748046875, 0.033447265625, -0.018798828125, -0.0308837890625, -0.0079345703125, -0.041748046875, -0.024658203125, -0.0277099609375, 0.0050048828125, -0.006011962890625, 0.041259765625, 0.00567626953125, 0.0029754638671875, -0.00018787384033203125, -0.00421142578125, -0.005523681640625, -0.0458984375, -0.011962890625, 0.059814453125, -0.0478515625, -0.030029296875, 0.0108642578125, -0.00958251953125, 0.00567626953125, 0.03662109375, 0.05615234375, -0.005615234375, 0.01123046875, 0.02392578125, -0.0244140625, -0.0029754638671875, 0.05419921875, -0.0059814453125, 0.0103759765625, 0.004547119140625, -0.01422119140625, 0.041748046875, 0.0751953125, 0.0106201171875, 0.01373291015625, -0.03466796875, -0.006866455078125, 0.06103515625, -0.01611328125, -0.0257568359375, -0.0230712890625, 0.04150390625, -0.0145263671875, 0.03564453125, 0.03466796875, -0.0018310546875, 0.055419921875, 5.078315734863281e-05, -0.041748046875, 0.00274658203125, 0.00177764892578125, -0.03857421875, -0.003662109375, 0.012451171875, -0.016845703125, -0.01708984375, 0.0198974609375, -0.040771484375, -0.010986328125, -0.01446533203125, -0.01300048828125, 0.007659912109375, 0.0113525390625, 0.1201171875, 0.000736236572265625, -0.001373291015625, 0.034423828125, -0.0235595703125, -0.0019073486328125, 0.0247802734375, -0.00177001953125, 0.031005859375, 0.044921875, -0.022216796875, 0.006744384765625, -0.052734375, 0.05615234375, -0.0264892578125, -0.042724609375, 0.0045166015625, -0.0185546875, -0.00616455078125, 0.046142578125, 0.0289306640625, 0.015869140625, 0.0081787109375, 0.018798828125, -0.0036163330078125, 0.005950927734375, 0.004547119140625, -0.039306640625, 0.0030059814453125, -0.037353515625, -0.02294921875, 0.022705078125, -0.0068359375, -0.03857421875, 0.019287109375, 0.0341796875, 0.0264892578125, 0.01611328125, 0.035888671875, -0.022705078125, 0.042236328125, -0.006744384765625, -0.0111083984375, -0.000946044921875, -0.029052734375, -0.05615234375, -0.0208740234375, 0.00885009765625, 0.036865234375, -0.036865234375, -0.052001953125, 0.04150390625, -0.05029296875, 0.026123046875, -0.054931640625, -0.039794921875, -0.0224609375, 0.043212890625, 0.0019989013671875, 0.00897216796875, -0.04638671875, 0.00066375732421875, 0.0306396484375, -0.03271484375, 0.053955078125, -0.048095703125, -0.0244140625, -0.044921875, -0.046142578125, -0.06982421875, 0.0279541015625, -0.0267333984375, -0.0238037109375, -0.0274658203125, 0.0242919921875, 0.007080078125, -0.0306396484375, -0.007659912109375, -0.04931640625, 0.02099609375, -0.02490234375, 0.0240478515625, 0.0157470703125, -0.0005035400390625, -0.007171630859375, 0.042236328125, 0.034912109375, 0.052001953125, 0.0185546875, 0.00738525390625, -0.034423828125, 0.03369140625, -0.05029296875, 0.01043701171875, -0.0145263671875, 0.005035400390625, 0.0654296875, -0.00811767578125, -0.056884765625, -0.03564453125, 0.0157470703125, -0.01190185546875, -0.025146484375, -0.01239013671875, 0.0184326171875, 0.0498046875, 0.03173828125, -0.024658203125, -0.008056640625, 0.040771484375, 0.01220703125, 0.0458984375, 0.0257568359375, 0.0673828125, -0.048828125, 0.01239013671875, 0.029296875, 0.007568359375, 0.00946044921875, 0.005615234375, 0.01214599609375, -0.016845703125, 0.037841796875, -0.03173828125, -0.0286865234375, 0.03564453125, -0.00193023681640625, 0.0155029296875, 0.025146484375, 0.00897216796875, 0.003692626953125, -0.0306396484375, 0.0206298828125, 0.035400390625, -0.029052734375, -0.0244140625, -0.0140380859375, -0.0101318359375, -0.01123046875, -0.004486083984375, -0.041259765625, -0.0224609375, -0.0284423828125, -0.028076171875, -0.0146484375, 0.03564453125, -0.059814453125, -0.038330078125, 0.01123046875, 0.01043701171875, -0.0269775390625, 0.000885009765625, -0.00848388671875, -0.0079345703125, -0.035400390625, -0.028076171875, -0.01202392578125, 0.005523681640625, 0.03515625, -0.0123291015625, -0.0267333984375, -0.00994873046875, 0.0186767578125, 0.06103515625, -0.00168609619140625, -0.0164794921875, -0.032470703125, 0.004364013671875, 0.031005859375, -0.031982421875, -0.052734375, 0.0186767578125, 0.00159454345703125, -0.01806640625, -0.0011138916015625, -0.0267333984375, -0.01806640625, 0.04296875, -0.0181884765625, -0.005615234375, -0.0181884765625, 0.00439453125, 0.024658203125, 0.01153564453125, 0.00555419921875, -0.00341796875, -0.006072998046875, 0.01373291015625, -0.029541015625, 0.01007080078125, -0.0296630859375, 0.0177001953125, 0.0019073486328125, 0.017333984375, -0.046142578125, -0.005096435546875, -0.0260009765625, 0.0111083984375, -0.0242919921875, 0.010498046875, 0.030029296875, -0.03662109375, 0.0198974609375, 0.0439453125, 0.01275634765625, -0.0224609375, -0.03466796875, -0.019287109375, 0.0184326171875, -0.0306396484375, -0.01220703125, -0.041748046875, -0.0177001953125, -0.01080322265625, 0.037841796875, 0.011962890625, -0.0302734375, 0.021240234375, 0.01275634765625, -0.0223388671875, -0.033203125, -0.0033416748046875, -0.035400390625, -0.03271484375, -0.0242919921875, -0.0081787109375, 0.04248046875, 0.0169677734375, 0.0296630859375, 0.048828125, 0.0194091796875, 0.0230712890625, -0.0045166015625, 0.004852294921875, 0.0203857421875, 0.0205078125, 0.0026702880859375, -0.0038299560546875, 0.01239013671875, 0.0001068115234375, -0.0208740234375, 0.0093994140625, -0.00860595703125, -0.07080078125, -0.0146484375, 0.0194091796875, 0.01007080078125, 0.009521484375, -0.0189208984375, 0.029541015625, 0.02490234375, -0.0216064453125, 0.00078582763671875, 0.02294921875, 0.0390625, -0.0177001953125, -0.06005859375, -0.005157470703125, -0.01287841796875, -0.046142578125, -0.0177001953125, 0.029296875, -0.00732421875, -0.01300048828125, 0.0079345703125, -0.00921630859375, -0.0145263671875, 0.008056640625, -0.00244140625, -0.00165557861328125, -0.01043701171875, 0.0032501220703125, -0.0009918212890625, -0.044677734375, 0.05908203125, -0.00897216796875, -0.036865234375, -0.006591796875, -0.038818359375, -0.0167236328125, 0.0244140625, -0.0146484375, -0.01611328125, -0.037841796875, 0.03759765625, 0.04443359375, 0.01953125, 0.00921630859375, -0.006439208984375, -0.00078582763671875, -0.0234375, 0.01275634765625, 0.0308837890625, -0.0145263671875, 0.0019073486328125, 0.03955078125, -0.0186767578125, 0.0255126953125, 0.007659912109375, 0.017578125, 0.0068359375, 0.000278472900390625, 0.0279541015625, -0.01318359375, 0.011962890625, 0.041015625, 0.027099609375, 0.0242919921875, -0.01153564453125, 0.01019287109375, -0.03564453125, -0.01434326171875, -0.039794921875, 0.00341796875, -0.0130615234375, 0.0081787109375, 0.0264892578125, 0.0230712890625, -0.0079345703125, 0.0279541015625, -0.0247802734375, 0.00787353515625, 0.052001953125, 0.0086669921875, 0.00787353515625, -0.000885009765625, -0.00162506103515625, 0.01287841796875, 0.021728515625, -0.008056640625, 0.00067138671875, 0.0260009765625, -0.01287841796875, 0.002685546875, -0.032470703125, -0.0157470703125, -0.004302978515625, 0.00010585784912109375, -0.0174560546875, 0.0267333984375, -0.01068115234375, 0.01068115234375, -0.018310546875, -0.0498046875, 0.042724609375, -0.0184326171875, -0.0028839111328125, -0.0306396484375, -0.0361328125, 0.0216064453125, -0.002105712890625, -0.030517578125, -0.0306396484375, 0.03369140625, -0.0172119140625, 0.0037384033203125, -0.022705078125, -0.01287841796875, 0.0238037109375, 0.03369140625, 0.031494140625, -0.0084228515625, 0.02392578125, 0.01318359375, 0.0269775390625, 0.018310546875, 0.01123046875, 0.0230712890625, -0.0390625, -0.017822265625, 0.00067138671875, 0.033935546875, 0.015869140625, 0.0033111572265625, -0.01177978515625, -0.0096435546875, 0.0244140625, 0.0054931640625, 0.01123046875, -0.0302734375, 5.4836273193359375e-06, 0.039794921875, -0.0184326171875, 0.028076171875, 0.0546875, -0.03173828125, -0.000823974609375, -0.0022430419921875, -0.0167236328125, -0.0546875, 0.009765625, -0.037841796875, 0.01019287109375, 0.005157470703125, -0.01019287109375, 0.023193359375, 0.0213623046875, -0.00537109375, 0.00109100341796875, -0.00946044921875, -0.0380859375, 0.0264892578125, -0.0024566650390625, -0.0279541015625, 0.0035552978515625, -0.022216796875, 0.03369140625, -0.0087890625, -0.010498046875, 0.006072998046875, 0.00970458984375, 0.006866455078125, -0.00174713134765625, 0.0277099609375, 0.0213623046875, 0.043701171875, -0.00146484375, 0.037109375, -0.0164794921875, -0.0001983642578125, 0.01080322265625, -0.007568359375, -0.024658203125, -0.00262451171875, -0.007476806640625, 0.0150146484375, -0.0115966796875, 0.0322265625, 0.00179290771484375, 0.0172119140625, -0.0220947265625, -0.02001953125, -0.036865234375, 0.052734375, -0.007568359375, -0.01019287109375, -0.021240234375, -0.003143310546875, -0.00946044921875, -0.0037078857421875, -0.0038299560546875, -0.0191650390625, 0.00506591796875, -0.0052490234375, -0.0299072265625, 0.012451171875, -0.033447265625, 0.03173828125, -0.01434326171875, 0.01080322265625, 0.00872802734375, -0.02490234375, 0.0086669921875, -0.01611328125, -0.01068115234375, 0.0159912109375, -0.0150146484375, 0.0150146484375, 0.021484375, -0.044189453125, -0.01385498046875, 0.0113525390625, -0.01068115234375, -0.0172119140625, -0.00051116943359375, 0.01904296875, -0.016357421875, -0.0152587890625, 0.0184326171875, 0.00811767578125, -0.0185546875, -0.0084228515625, -0.0361328125, -0.0238037109375, -0.01275634765625, 0.022705078125, 0.003143310546875, 0.00738525390625, -0.02734375, 0.00274658203125, 0.01104736328125, -0.002655029296875, -0.0264892578125, -0.040771484375, 0.00396728515625, 0.0026702880859375, -0.00921630859375, 0.000568389892578125, 0.00396728515625, 0.00970458984375, 0.0322265625, 0.00421142578125, -0.046142578125, 0.032470703125, -0.00023651123046875, 0.003265380859375, 0.02392578125, -0.029296875, 0.01348876953125, 0.016845703125, -0.023193359375, 0.003692626953125, 0.043212890625, -0.0027923583984375, 0.017333984375, 0.00897216796875, -0.0084228515625, -0.019287109375, -0.00060272216796875, 0.0296630859375, 0.01055908203125, -0.01483154296875, 0.00634765625, -0.0181884765625, 4.649162292480469e-05, -0.000675201416015625, 0.033447265625, -0.046142578125, -0.0279541015625, -0.00127410888671875, -0.0242919921875, 0.04638671875, 0.0023040771484375, -0.046142578125, 0.06640625, 0.0279541015625, 0.01324462890625, 0.017822265625, -0.02001953125, 0.01190185546875, -0.008544921875, -0.0024871826171875, -0.0186767578125, 0.027099609375, 9.1552734375e-05, -0.00091552734375, -0.006683349609375, 0.017333984375, 0.0101318359375, -0.01300048828125, -0.013671875, -0.0115966796875, 0.006591796875, -0.0162353515625, -0.0263671875, 0.00994873046875, -0.04931640625, 0.0289306640625, 0.039794921875, -0.01416015625, -0.026123046875, 0.03564453125, -0.024658203125, 0.01544189453125, -0.006927490234375, 0.01531982421875, 0.002197265625, 0.01025390625, -0.0230712890625, 0.01092529296875, 0.01177978515625, 0.025390625, -0.003631591796875, -0.0186767578125, 0.021484375, -0.055908203125, 0.03369140625, 0.02734375, -0.02197265625, 0.0191650390625, 0.022705078125, 0.0023040771484375, -0.0257568359375, 0.0103759765625, 0.0255126953125, -0.0034637451171875, 0.003662109375, 0.0140380859375, -0.0084228515625, 0.054931640625, 0.03662109375, -0.01177978515625, 0.02392578125, 0.00616455078125, 0.0135498046875, 0.01336669921875, 0.01422119140625, 0.000812530517578125, -0.03271484375, 0.01043701171875, 0.006500244140625, 0.0184326171875, -0.00177001953125, -0.0005950927734375, 0.00811767578125, 0.00946044921875, 0.0274658203125, 0.00131988525390625, -0.0003795623779296875, -0.01324462890625, 0.0216064453125, 0.0084228515625, 0.054931640625, -0.021240234375, 0.0242919921875, -0.0086669921875, 0.05224609375, -0.01007080078125, 0.046875, 0.00738525390625, 0.057373046875, 0.0260009765625, -0.0174560546875, 0.0123291015625, -0.01446533203125, 0.00616455078125, -0.00238037109375, 0.0264892578125, -0.0027313232421875, 0.0166015625, -0.00970458984375, 0.0096435546875, 0.01287841796875, ...]}], 'usage': {'prompt_tokens': 9, 'total_tokens': 9}}启动 xinference 设置环境变量：
XINFERENCE_MODEL_SRC=modelscope HF_ENDPOINT=https://hf-mirror.com xinference-local
启动之后ui页面中有个地方显示地址：0.0.0.0:xxx,，这个xxx端口哪里来的？
https://inference.readthedocs.io/zh-cn/latest/models/model_abilities/embed.html
先读文档
我从0 开始加载这个模型，没有问题，调用也正常。
In [1]: from xinference.client import Client In [2]: client = Client('http://gpu:36666') In [3]: model = client.get_model('jina-embeddings-v3') In [4]: model.create_embedding("What is the capital of China?") Out[4]: {'object': 'list', 'model': 'jina-embeddings-v3', 'model_replica': 'jina-embeddings-v3-0', 'data': [{'index': 0, 'object': 'embedding', 'embedding': [0.052001953125, -0.005218505859375, ...]}], 'usage': {'prompt_tokens': 9, 'total_tokens': 9}}启动 xinference 设置环境变量：
XINFERENCE_MODEL_SRC=modelscope HF_ENDPOINT=https://hf-mirror.com xinference-local
纯内网运行xinference 向量模型jina-embeddings-v3手动从modelscope下载 运行向量模型时，手动指定模型文件路径
启动过程中，仍然要去互联网请求资源，导致启动失败。 @qinxuye
启动报错日志：
2025-07-22 05:19:27,468 xinference.core.supervisor 4284 DEBUG Enter launch_builtin_model, model_uid: jina-embeddings-v3, model_name: jina-embeddings-v3, model_size: , model_format: None, quantization: None, replica: 1, enable_xavier: False, kwargs: {}
2025-07-22 05:19:27,470 xinference.core.worker 4284 DEBUG Enter get_model_count, args: <xinference.core.worker.WorkerActor object at 0x7f7e2a729850>, kwargs:
2025-07-22 05:19:27,470 xinference.core.worker 4284 DEBUG Leave get_model_count, elapsed time: 0 s
2025-07-22 05:19:27,471 xinference.core.worker 4284 INFO [request 70227d5e-66bb-11f0-9dc4-dad7fe94c27e] Enter launch_builtin_model, args: <xinference.core.worker.WorkerActor object at 0x7f7e2a729850>, kwargs: model_uid=jina-embeddings-v3-0,model_name=jina-embeddings-v3,model_size_in_billions=None,model_format=None,quantization=None,model_engine=sentence_transformers,model_type=embedding,n_gpu=auto,request_limits=None,peft_model_config=None,gpu_idx=[5],download_hub=None,model_path=/data/models/jina-embeddings-v3/,xavier_config=None
2025-07-22 05:19:27,472 xinference.core.worker 4284 INFO You specify to launch the model: jina-embeddings-v3 on GPU index: [5] of the worker: 0.0.0.0:46478, xinference will automatically ignore the `n_gpu` option.
2025-07-22 05:19:27,473 xinference.core.worker 4284 WARNING WARNING!!! GPU index 5 has been occupied with these models on it: ['Qwen3-Reranker-4B-0']
Actor caller has created too many clients (220 >= 100), the global router may not be set.
2025-07-22 05:19:28,205 xinference.core.progress_tracker 4284 DEBUG Setting progress, request id: launching-jina-embeddings-v3-0, progress: 0.0
2025-07-22 05:19:28,206 xinference.model.embedding.embed_family 4284 DEBUG Embedding model jina-embeddings-v3 found in ModelScope.
2025-07-22 05:19:28,209 xinference.core.progress_tracker 4284 DEBUG Setting progress, request id: launching-jina-embeddings-v3-0, progress: 0.8
2025-07-22 05:19:28,212 xinference.core.progress_tracker 4284 DEBUG Setting progress, request id: launching-jina-embeddings-v3-0, progress: 0.8
2025-07-22 05:19:30,002 transformers.utils.import_utils 622335 DEBUG Detected accelerate version: 1.8.1
Detected accelerate version: 1.8.1
2025-07-22 05:19:30,004 transformers.utils.import_utils 622335 DEBUG Detected jinja2 version: 3.1.6
Detected jinja2 version: 3.1.6
2025-07-22 05:19:30,004 transformers.utils.import_utils 622335 DEBUG Detected openai version: 1.90.0
Detected openai version: 1.90.0
2025-07-22 05:19:30,008 transformers.utils.import_utils 622335 DEBUG Detected pandas version: 2.3.1
Detected pandas version: 2.3.1
2025-07-22 05:19:30,009 transformers.utils.import_utils 622335 DEBUG Detected peft version: 0.16.0
Detected peft version: 0.16.0
2025-07-22 05:19:30,009 transformers.utils.import_utils 622335 DEBUG Detected psutil version: 7.0.0
Detected psutil version: 7.0.0
2025-07-22 05:19:30,010 transformers.utils.import_utils 622335 DEBUG Detected pygments version: 2.19.2
Detected pygments version: 2.19.2
2025-07-22 05:19:30,010 transformers.utils.import_utils 622335 DEBUG Detected safetensors version: 0.5.3
Detected safetensors version: 0.5.3
2025-07-22 05:19:30,013 transformers.utils.import_utils 622335 DEBUG Detected scipy version: 1.15.3
Detected scipy version: 1.15.3
2025-07-22 05:19:30,013 transformers.utils.import_utils 622335 DEBUG Detected sentencepiece version: 0.2.0
Detected sentencepiece version: 0.2.0
2025-07-22 05:19:30,014 transformers.utils.import_utils 622335 DEBUG Detected gguf version: 0.17.1
Detected gguf version: 0.17.1
2025-07-22 05:19:30,015 transformers.utils.import_utils 622335 DEBUG Detected timm version: 1.0.17
Detected timm version: 1.0.17
2025-07-22 05:19:30,016 transformers.utils.import_utils 622335 DEBUG Detected tokenizers version: 0.21.2
Detected tokenizers version: 0.21.2
2025-07-22 05:19:30,016 transformers.utils.import_utils 622335 DEBUG Detected torchaudio version: 2.7.0
Detected torchaudio version: 2.7.0
2025-07-22 05:19:30,016 transformers.utils.import_utils 622335 DEBUG Detected torchvision version: 0.22.0
Detected torchvision version: 0.22.0
2025-07-22 05:19:30,017 transformers.utils.import_utils 622335 DEBUG Detected tiktoken version: 0.9.0
Detected tiktoken version: 0.9.0
2025-07-22 05:19:30,017 transformers.utils.import_utils 622335 DEBUG Detected triton version: 3.3.0
Detected triton version: 3.3.0
2025-07-22 05:19:30,018 transformers.utils.import_utils 622335 DEBUG Detected rich version: 14.0.0
Detected rich version: 14.0.0
2025-07-22 05:19:30,018 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,036 transformers.utils.import_utils 622335 DEBUG Detected PIL version 11.3.0
Detected PIL version 11.3.0
2025-07-22 05:19:30,279 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,280 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,281 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,282 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,284 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,285 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,286 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,287 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,289 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:30,290 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:31,572 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
2025-07-22 05:19:31,583 transformers.utils.import_utils 622335 DEBUG Detected torch version: 2.7.0
Detected torch version: 2.7.0
INFO 07-22 05:19:31 [__init__.py:244] Automatically detected platform cuda.
2025-07-22 05:19:35,259 xinference.core.model 622335 DEBUG Starting ModelActor at 0.0.0.0:33991, uid: b'jina-embeddings-v3-0'
2025-07-22 05:19:35,259 xinference.core.model 622335 INFO Start requests handler.
2025-07-22 05:19:35,489 transformers.configuration_utils 622335 INFO loading configuration file /data/models/jina-embeddings-v3/config.json
loading configuration file /data/models/jina-embeddings-v3/config.json
2025-07-22 05:20:55,579 transformers.dynamic_module_utils 622335 INFO Could not locate the configuration_xlm_roberta.py inside jinaai/xlm-roberta-flash-implementation.
Could not locate the configuration_xlm_roberta.py inside jinaai/xlm-roberta-flash-implementation.
2025-07-22 05:20:55,586 xinference.core.worker 4284 ERROR Failed to load model jina-embeddings-v3-0
huggingface_hub.errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/worker.py", line 1114, in launch_builtin_model
await model_ref.load()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
File "xoscar/core.pyx", line 564, in __on_receive__
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
async with self._lock:
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
result = await result
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/model.py", line 476, in load
await asyncio.to_thread(self._model.load)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/model/embedding/sentence_transformers/core.py", line 115, in load
self._model = SentenceTransformer(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 327, in __init__
modules, self.module_kwargs = self._load_sbert_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 2235, in _load_sbert_model
module = module_class(model_name_or_path, **common_transformer_init_kwargs)
File "/data/xinference/openmind_hub/huggingface/modules/transformers_modules/custom_st.py", line 66, in __init__
self.config = AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1211, in from_pretrained
config_class = get_class_from_dynamic_module(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 570, in get_class_from_dynamic_module
final_module = get_cached_module_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 372, in get_cached_module_file
resolved_module_file = cached_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 312, in cached_file
file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 543, in cached_files
raise OSError(
OSError: [address=0.0.0.0:33991, pid=622335] We couldn't connect to 'https://hf-mirror.com' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
2025-07-22 05:20:55,590 xinference.core.progress_tracker 4284 DEBUG Setting progress, request id: launching-jina-embeddings-v3-0, progress: 1.0
2025-07-22 05:20:55,693 xinference.core.worker 4284 ERROR [request 70227d5e-66bb-11f0-9dc4-dad7fe94c27e] Leave launch_builtin_model, error: [address=0.0.0.0:33991, pid=622335] We couldn't connect to 'https://hf-mirror.com' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'., elapsed time: 88 s
huggingface_hub.errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/utils.py", line 93, in wrapped
ret = await func(*args, **kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/worker.py", line 1114, in launch_builtin_model
await model_ref.load()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
File "xoscar/core.pyx", line 564, in __on_receive__
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
async with self._lock:
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
result = await result
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/model.py", line 476, in load
await asyncio.to_thread(self._model.load)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/model/embedding/sentence_transformers/core.py", line 115, in load
self._model = SentenceTransformer(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 327, in __init__
modules, self.module_kwargs = self._load_sbert_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 2235, in _load_sbert_model
module = module_class(model_name_or_path, **common_transformer_init_kwargs)
File "/data/xinference/openmind_hub/huggingface/modules/transformers_modules/custom_st.py", line 66, in __init__
self.config = AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1211, in from_pretrained
config_class = get_class_from_dynamic_module(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 570, in get_class_from_dynamic_module
final_module = get_cached_module_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 372, in get_cached_module_file
resolved_module_file = cached_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 312, in cached_file
file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 543, in cached_files
raise OSError(
OSError: [address=0.0.0.0:33991, pid=622335] We couldn't connect to 'https://hf-mirror.com' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
2025-07-22 05:20:55,695 xinference.core.supervisor 4284 DEBUG [request a4b878ca-66bb-11f0-9dc4-dad7fe94c27e] Enter terminate_model, args: <xinference.core.supervisor.SupervisorActor object at 0x7f7e2abfe980>,jina-embeddings-v3, kwargs: suppress_exception=True
2025-07-22 05:20:55,696 xinference.core.supervisor 4284 DEBUG [request a4b878ca-66bb-11f0-9dc4-dad7fe94c27e] Leave terminate_model, elapsed time: 0 s
2025-07-22 05:20:55,706 xinference.api.restful_api 3668 ERROR [address=0.0.0.0:33991, pid=622335] We couldn't connect to 'https://hf-mirror.com' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
huggingface_hub.errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/api/restful_api.py", line 1077, in launch_model
model_uid = await (await self._get_supervisor_ref()).launch_builtin_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
File "xoscar/core.pyx", line 564, in __on_receive__
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
async with self._lock:
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
result = await result
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/supervisor.py", line 1195, in launch_builtin_model
await _launch_model()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/supervisor.py", line 1130, in _launch_model
subpool_address = await _launch_one_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/supervisor.py", line 1084, in _launch_one_model
subpool_address = await worker_ref.launch_builtin_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
File "xoscar/core.pyx", line 564, in __on_receive__
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
async with self._lock:
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
result = await result
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/utils.py", line 93, in wrapped
ret = await func(*args, **kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/worker.py", line 1114, in launch_builtin_model
await model_ref.load()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
File "xoscar/core.pyx", line 564, in __on_receive__
raise ex
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
async with self._lock:
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
with debug_async_timeout('actor_lock_timeout',
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
result = await result
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/core/model.py", line 476, in load
await asyncio.to_thread(self._model.load)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/xinference/model/embedding/sentence_transformers/core.py", line 115, in load
self._model = SentenceTransformer(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 327, in __init__
modules, self.module_kwargs = self._load_sbert_model(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 2235, in _load_sbert_model
module = module_class(model_name_or_path, **common_transformer_init_kwargs)
File "/data/xinference/openmind_hub/huggingface/modules/transformers_modules/custom_st.py", line 66, in __init__
self.config = AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1211, in from_pretrained
config_class = get_class_from_dynamic_module(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 570, in get_class_from_dynamic_module
final_module = get_cached_module_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 372, in get_cached_module_file
resolved_module_file = cached_file(
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 312, in cached_file
file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
File "/data/miniconda3/envs/xinference1.7.1.post1/lib/python3.10/site-packages/transformers/utils/hub.py", line 543, in cached_files
raise OSError(
OSError: [address=0.0.0.0:33991, pid=622335] We couldn't connect to 'https://hf-mirror.com' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
