huggingsound icon indicating copy to clipboard operation
huggingsound copied to clipboard

Issue with converting large dataset_from_dict_list

Open rovr opened this issue 1 year ago • 0 comments

The following block slows down with the larger dict lists and becomes completely unusable when it contains millions of items:

https://github.com/jonatasgrosman/huggingsound/blob/8178f8e5b332bdc2a6a2f36431a5f66f6cb10d05/huggingsound/utils.py#L72-L76

changing it to

    keys = data[0].keys()
    transformed_data = {key: [d[key] for d in data] for key in keys}

fixes the issue

rovr avatar Mar 30 '23 19:03 rovr