databroker
databroker copied to clipboard
Fix dtype deserialization
Closes #820
Okay so not only the incoming dtype needs to be deserialized correctly, but in some cases also the structured arrays. So the changes needed are a little more extensive.
- Shape Validation (
mongo): https://github.com/danielballan/databroker/blob/6c7c26a8752709ee102e2551c2a12bd4d671a9f2/databroker/mongo_normalized.py#L932-L946
numpy.asarray(item): the conversion of item to an ndarray has to be done with the appropriate numpy dtype, from the data_key['dtype_descr'] if it exists, and each structured array element has to be turned into a tuple
- Config xarray: https://github.com/danielballan/databroker/blob/6c7c26a8752709ee102e2551c2a12bd4d671a9f2/databroker/mongo_normalized.py#L1081
columns[key] = numpy.array(column, dtype=numpy_dtype)
if numpy_dtype is a structured array, the data in column needs to be turned into a list of lists of tuples.
How would you like to proceed, sould I open my own PR or do you want to implement these changes?