weaviate-python-client icon indicating copy to clipboard operation
weaviate-python-client copied to clipboard

Published 20 hours ago • verified publisher icon weaviate

insert_many silently discards vectors if they're NumPy arrays

Open kylrth opened this issue 1 year ago • 4 comments
trafficstars

If I have a collection c and call c.data.insert_many on a list[DataObject], each of which with the vector field set to a NumPy array, the upload returns without error but silently discards the vectors. The line that causes this behavior is here.

I would expect the NumPy array to be interpreted as a list of floats. Or, if this is undesirable for some reason, passing a NumPy array as a vector should result in an error.

kylrth avatar Apr 10 '24 02:04 kylrth

hi @kylrth, this is definitely a bug since we handle NumPy arrays gracefully in other areas of the API. I will make sure it's fixed for the next release. Thanks for raising it!

tsmith023 avatar Apr 11 '24 08:04 tsmith023

hi @tsmith023 Can I work on this?

manu156 avatar Apr 18 '24 18:04 manu156

Yes thank you, that would be great!

tsmith023 avatar Apr 18 '24 18:04 tsmith023

@tsmith023 I have raised a PR, please review it. I was not sure how to do it without importing numpy, I was having troubles with Pydantic throwing errors when specifying types without importing them. Also numpy.ndarray doesn't seem to implement Sequence from ABC, only closest thing it implements is Iterable, So I added a custom class for validation.

manu156 avatar Apr 20 '24 09:04 manu156