milvus
milvus copied to clipboard
[Bug]: [JSON] Json field in Output_fields returns bytes format value
Is there an existing issue for this?
- [X] I have searched the existing issues
Environment
- Milvus version: master-20230602-6b1712e0
- Deployment mode(standalone or cluster): both
- MQ type(rocksmq, pulsar or kafka): all
- SDK version(e.g. pymilvus v2.0.0rc2): 2.4.0.dev67
- OS(Ubuntu or CentOS):
- CPU/Memory:
- GPU:
- Others:
Current Behavior
Json field in Output_fields returns bytes format value
>>> res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "json_field[\"number\"] <= 10", output_fields=["json_field"])
>>>
>>> res[0][0]
id: 0, distance: 0.0, entity: {'json_field': b'{"number":0,"float":1.1111111111111112,"int64":1000000000000000180980893038800089,"string":"milvus","bool":true,"list":[],"json":{"number":0,"string":"milvus","bool":true}}'}
>>>
Expected Behavior
res[0][0] id: 441895033600555812, distance: 18.436702728271484, entity: {'json_field': {'number': 3, 'string': '3', 'bool': True, 'list': [0, 1, 2]}}
Steps To Reproduce
from pymilvus import CollectionSchema, FieldSchema
from pymilvus import Collection
from pymilvus import connections
from pymilvus import DataType
from pymilvus import Partition
from pymilvus import utility
connections.connect()
dim = 128
int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True)
float_field = FieldSchema(name="float", dtype=DataType.FLOAT)
bool_field = FieldSchema(name="bool", dtype=DataType.BOOL)
string_field = FieldSchema(name="string", dtype=DataType.VARCHAR, max_length=65535)
json_field = FieldSchema(name="json_field", dtype=DataType.JSON)
json_field_1 = FieldSchema(name="json_field_1", dtype=DataType.JSON)
float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
schema = CollectionSchema(fields=[int64_field, float_field, bool_field, json_field, json_field_1, float_vector])
collection = Collection("test_search_collection_multiple_json_0", schema=schema)
import random
nb=3000
vectors = [[random.random() for _ in range(dim)] for _ in range(nb)]
import numpy as np
res = collection.insert([[i for i in range(nb)], [np.float32(i) for i in range(nb)], [np.bool_(i) for i in range(nb)], [{"number": i, "float": 1.111111111111111111111111111, "int64": 1000000000000000180980893038800089, "string": "milvus", "bool": True, "list": [j for j in range(0, i)], "json": {"number": i, "string": "milvus", "bool": True}} for i in range(0, nb)], [{"number": i, "string": "milvus", "bool": True, "list": [j for j in range(0, i)], "json": {"number": i, "string": "milvus", "bool": True}} for i in range(0, nb)], vectors])
index_param = {"index_type": "IVF_SQ8", "metric_type": "L2", "params": {"nlist": 8192}}
collection.create_index("float_vector", index_param, index_name="index_name_1")
collection.load()
default_search_params = {"metric_type": "L2", "params": {"nprobe": 10}}
limit = 10
nq =1
res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "json_field[\"number\"] <= 10", output_fields=["json_field"])
Milvus Log
No response
Anything else?
No response
When create enabled dynamic collection, the result is: (not bytes format)
>>> res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "float in [1, 2, 3]", output_fields=["json_field"])
>>> res[0][0]
id: 441895033600555812, distance: 18.436702728271484, entity: {'json_field': {'number': 3, 'string': '3', 'bool': True, 'list': [0, 1, 2]}}
@czs007 Could you please have a look? thanks.
/assign @czs007
/unassign @xiaocai2333
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen
.
Verified and fixed. pymilvus: 2.4.0.dev81 milvus: master-20230711-96c987ed
>>> res[0][0]
id: 0, distance: 0.0, entity: {'json_field': {'number': 0, 'float': 1.1111111111111112, 'int64': 1000000000000000180980893038800089, 'string': 'milvus', 'bool': True, 'list': [], 'json': {'number': 0, 'string': 'milvus', 'bool': True}}}