cottoncandy icon indicating copy to clipboard operation
cottoncandy copied to clipboard

Published 20 hours ago • verified publisher icon gallantlab

Implement fast NPY upload

Open spopham opened this issue 7 years ago • 2 comments

Rewrite upload_npy_array and download_npy_array to do in-memory and have a npz compressed option

Reference: https://github.com/numpy/numpy/blob/master/numpy/lib/format.py

spopham avatar Aug 02 '17 18:08 spopham

👍

anwarnunez avatar Aug 02 '17 18:08 anwarnunez

Just had a quick look b/c it was on my mind. This should be pretty straightforward (see below). This should really only be allowed for uncompressed arrays and GZIP compressed arrays.

https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L326 https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L430

Using these those two functions, I think one just needs to dump that at the beginning of the StringIO (https://github.com/gallantlab/cottoncandy/blob/df64302ccbc7c00571a165f8ad41a5af1bde6a47/cottoncandy/interfaces.py#L631 and https://github.com/gallantlab/cottoncandy/blob/df64302ccbc7c00571a165f8ad41a5af1bde6a47/cottoncandy/interfaces.py#L624)

anwarnunez avatar Mar 17 '20 20:03 anwarnunez