[BUG] Cannot create an OBJECT array from memory buffer

[davidbuniat]

🐛🐛 Bug Report

⚗️ Current Behavior

Cannot create an OBJECT array from memory buffer. Dataset has been created using numpy objects and a tensor dtype is

Input Code

> ds = hub.load("hub://activeloop/abalone_full_dataset")
> ds.Sex.numpy()
ValueError: cannot create an OBJECT array from memory buffer
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-29-f439a6f63b90> in <module>
----> 1 ds.Sex.numpy()
~/.local/lib/python3.6/site-packages/hub/core/tensor.py in numpy(self, aslist)
    463         """
    464 
--> 465         return self.chunk_engine.numpy(self.index, aslist=aslist)
    466 
    467     def __str__(self):
~/.local/lib/python3.6/site-packages/hub/core/chunk_engine.py in numpy(self, index, aslist, use_data_cache)
    777                         global_sample_index
    778                     )
--> 779                     sample = chunk.read_sample(local_sample_index)[
    780                         tuple(entry.value for entry in index.values[1:])
    781                     ]
~/.local/lib/python3.6/site-packages/hub/core/chunk/uncompressed_chunk.py in read_sample(self, local_index, cast, copy)
     78             return bytes_to_text(buffer, self.htype)
     79         buffer = bytes(buffer) if copy else buffer
---> 80         return np.frombuffer(buffer, dtype=self.dtype).reshape(shape)
     81 
     82     def update_sample(self, local_index: int, sample: InputSample):
ValueError: cannot create an OBJECT array from memory buffer

⚙️ Environment

• Python version(s): Python 3.6.9
• OS: Ubuntu 18.04.6 LTS
• IDE: VS-Code

🧰 Possible Solution

Either do not allow numpy objects to be uploaded or parse numpy() properly.

[rajdeepdas2000]

I am interested to work on this issue.

[tatevikh]

Hi @rajdeepdas2000 ! Sure, go for it:)

[h20200051]

@davidbuniat I would like to work under this issue.Kindly assign me

[rajdeepdas2000]

The abalone_full_dataset is not available in hub(deeplake). What is the way to import the dataset from a csv file?

[davidbuniat]

@rajdeepdas2000 @h20200051 really appreciate willingness to contribute here, but just checked and this problem has been fixed. I will close it for now and can be reopened if the problem still persists.

[rajdeepdas2000]

Alright. But can you tell how do we import datasets in deeplake from our local machine?

[davidbuniat]

you can use local paths such as ./path/to/your/local/dataset while creating or loading a dataset.

import deeplake
ds = deeplake.empty("./path/to/your/local/dataset")
...

and then access it

ds = deeplake.load("./path/to/your/local/dataset")