milvus icon indicating copy to clipboard operation
milvus copied to clipboard

[Bug]: [JSON] Json field in Output_fields returns bytes format value

Open binbinlv opened this issue 1 year ago • 1 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Environment

- Milvus version: master-20230602-6b1712e0
- Deployment mode(standalone or cluster): both
- MQ type(rocksmq, pulsar or kafka):    all
- SDK version(e.g. pymilvus v2.0.0rc2): 2.4.0.dev67
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Json field in Output_fields returns bytes format value

>>> res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "json_field[\"number\"] <= 10", output_fields=["json_field"])
>>>
>>> res[0][0]
id: 0, distance: 0.0, entity: {'json_field': b'{"number":0,"float":1.1111111111111112,"int64":1000000000000000180980893038800089,"string":"milvus","bool":true,"list":[],"json":{"number":0,"string":"milvus","bool":true}}'}
>>>

Expected Behavior

res[0][0] id: 441895033600555812, distance: 18.436702728271484, entity: {'json_field': {'number': 3, 'string': '3', 'bool': True, 'list': [0, 1, 2]}}

Steps To Reproduce

from pymilvus import CollectionSchema, FieldSchema
from pymilvus import Collection
from pymilvus import connections
from pymilvus import DataType
from pymilvus import Partition
from pymilvus import utility
connections.connect()

dim = 128
int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True)
float_field = FieldSchema(name="float", dtype=DataType.FLOAT)
bool_field = FieldSchema(name="bool", dtype=DataType.BOOL)
string_field = FieldSchema(name="string", dtype=DataType.VARCHAR, max_length=65535)
json_field = FieldSchema(name="json_field", dtype=DataType.JSON)
json_field_1 = FieldSchema(name="json_field_1", dtype=DataType.JSON)
float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
schema = CollectionSchema(fields=[int64_field, float_field, bool_field, json_field, json_field_1, float_vector])
collection = Collection("test_search_collection_multiple_json_0", schema=schema)
import random
nb=3000
vectors = [[random.random() for _ in range(dim)] for _ in range(nb)]
import numpy as np
res = collection.insert([[i for i in range(nb)], [np.float32(i) for i in range(nb)], [np.bool_(i) for i in range(nb)], [{"number": i, "float": 1.111111111111111111111111111, "int64":  1000000000000000180980893038800089, "string": "milvus", "bool": True, "list": [j for j in range(0, i)], "json": {"number": i, "string": "milvus", "bool": True}} for i in range(0, nb)], [{"number": i, "string": "milvus", "bool": True, "list": [j for j in range(0, i)], "json": {"number": i, "string": "milvus", "bool": True}} for i in range(0, nb)], vectors])
index_param = {"index_type": "IVF_SQ8", "metric_type": "L2", "params": {"nlist": 8192}}
collection.create_index("float_vector", index_param, index_name="index_name_1")
collection.load()
default_search_params = {"metric_type": "L2", "params": {"nprobe": 10}}
limit = 10
nq =1
res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "json_field[\"number\"] <= 10", output_fields=["json_field"])

Milvus Log

No response

Anything else?

No response

binbinlv avatar Jun 02 '23 11:06 binbinlv

When create enabled dynamic collection, the result is: (not bytes format)

>>> res = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "float in [1, 2, 3]", output_fields=["json_field"])
>>> res[0][0]
id: 441895033600555812, distance: 18.436702728271484, entity: {'json_field': {'number': 3, 'string': '3', 'bool': True, 'list': [0, 1, 2]}}

binbinlv avatar Jun 02 '23 11:06 binbinlv

@czs007 Could you please have a look? thanks.

binbinlv avatar Jun 05 '23 02:06 binbinlv

/assign @czs007

binbinlv avatar Jun 05 '23 02:06 binbinlv

/unassign @xiaocai2333

binbinlv avatar Jun 05 '23 02:06 binbinlv

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Jul 09 '23 07:07 stale[bot]

Verified and fixed. pymilvus: 2.4.0.dev81 milvus: master-20230711-96c987ed

>>> res[0][0]
id: 0, distance: 0.0, entity: {'json_field': {'number': 0, 'float': 1.1111111111111112, 'int64': 1000000000000000180980893038800089, 'string': 'milvus', 'bool': True, 'list': [], 'json': {'number': 0, 'string': 'milvus', 'bool': True}}}

binbinlv avatar Jul 12 '23 04:07 binbinlv