milvus
milvus copied to clipboard
milvus-io
[Bug]: Can not find pymilvus.model.hybrid
Is there an existing issue for this?
- [X] I have searched the existing issues
Environment
- Milvus version: 2.4.0
- Deployment mode(standalone or cluster): Lite
- MQ type(rocksmq, pulsar or kafka):
- SDK version(e.g. pymilvus v2.0.0rc2): Latest (pip install -U pymilvus, pip install -U pymilvus[model])
- OS(Ubuntu or CentOS): Ubuntu WSL2
- CPU/Memory:
- GPU: No
- Others:
Current Behavior
Can not import from pymilvus.model.hybrid
Expected Behavior
This should work from pymilvus.model.hybrid import BGEM3EmbeddingFunction
Steps To Reproduce
No response
Milvus Log
No response
Anything else?
No response
Yes, for more information, I did pip install -U pymilvus pip install -U pymilvus[model]
@yanliang567: GitHub didn't allow me to assign the following users: wxywb.
Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide
In response to this:
/assign @wxywb /unassign
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@vuminhquang: GitHub didn't allow me to assign the following users: wxywb.
Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide
In response to this:
/assign @wxywb
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@vuminhquang , can you run
pip list |grep milvus-model
to show your model library status?
i am able to import from pymilvus.model.hybrid import BGEM3EmbeddingFunction
my package version
pip list | grep milvus-model
milvus-model 0.2.0
however, i had this error when running code block:
docs = [
"Artificial intelligence was founded as an academic discipline in 1956.",
"Alan Turing was the first person to conduct substantial research in AI.",
"Born in Maida Vale, London, Turing was raised in southern England.",
]
docs_embeddings = bge_m3_ef.encode_documents(docs)
print("Embeddings:", docs_embeddings)
print("Dense document dim:", bge_m3_ef.dim["dense"], docs_embeddings["dense"][0].shape)
print("Sparse document dim:", bge_m3_ef.dim["sparse"], list(docs_embeddings["sparse"])[0].shape)
>
Embeddings: {'dense': [array([-0.02505942, -0.00142191, 0.04015464, ..., -0.02094925,
0.02623658, 0.00324097], dtype=float32), array([ 0.00118462, 0.00649289, -0.00735763, ..., -0.01446299,
0.0424368 , -0.0179482 ], dtype=float32), array([ 0.00415286, -0.01014927, 0.00098102, ..., -0.02559664,
0.0808467 , 0.00141649], dtype=float32)], 'sparse': <3x250002 sparse matrix of type '<class 'numpy.float32'>'
with 43 stored elements in COOrdinate format>}
Dense document dim: 1024 (1024,)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[<ipython-input-11-c15f5f76006f>](https://localhost:8080/#) in <cell line: 11>()
9 print("Embeddings:", docs_embeddings)
10 print("Dense document dim:", bge_m3_ef.dim["dense"], docs_embeddings["dense"][0].shape)
---> 11 print("Sparse document dim:", bge_m3_ef.dim["sparse"], list(docs_embeddings["sparse"])[0].shape)
[/usr/local/lib/python3.10/dist-packages/scipy/sparse/_base.py](https://localhost:8080/#) in __iter__(self)
250 def __iter__(self):
251 for r in range(self.shape[0]):
--> 252 yield self[r, :]
253
254 def _getmaxprint(self):
TypeError: 'coo_matrix' object is not subscriptable
Is that error on list(docs_embeddings["sparse"]) ?
pip list | grep scipy
scipy 1.11.4
Hi @wxywb, This is my result pip list | grep milvus-model milvus-model 0.2.0
@khanhj it is supposed to be a csr_array instead of coo_matrix, It seems caused by version of scipy, I will fixed it for next release.
from scipy.sparse import csr_array
res = list(csr_array(docs_embeddings['sparse']))[0]
this should work for now.
@vuminhquang please tell me your python version
- check the milvus-model functionality
from milvus_model.hybrid import BGEM3EmbeddingFunction
- if milvus-model test passed, please try to run
from pymilvus.model.hybrid import BGEM3EmbeddingFunction
and show me full stacktrace showed in your terminal.
@wxywb The python version is: 3.11.9 This is my check:
- from milvus_model.hybrid import BGEM3EmbeddingFunction: OK
- from pymilvus.model.hybrid import BGEM3EmbeddingFunction: Not passed. There is not hybrid.
@vuminhquang ,could you run your code , and show information (some error or exceptions) in your bash? It is not surprised your IDE can not find correct symbols of modules in pymilvus.model, because it used an import trick that import hybrid, dense or sparse when executing.
Both can be import from console @wxywb, If I want to write the code directly from IDE, (not care on the beautiful of code), which class should I call directly? Is milvus_model.hybrid the way to use? Thanks.
Both can be import from console @wxywb, If I want to write the code directly from IDE, (not care on the beautiful of code), which class should I call directly? Is milvus_model.hybrid the way to use? Thanks.
yes in this case, you should use milvus_model instead of pymilvus.model, we have a separate repo for it https://github.com/milvus-io/milvus-model