Alexander Clausen

So, actually....  Pickling is slow, but image encoding takes the crown. Even applying the colormap is slower than pickling. Just as a preliminary result, still need to verify by...

Just as a note, the recently released numpy 1.16 now supports the new pickle protocol; see https://github.com/numpy/numpy/releases/tag/v1.16.0

A WIP fix is contained in https://github.com/LiberTEM/LiberTEM/pull/1159

Could be interesting/useful: [sparse support in pytorch](https://pytorch.org/docs/stable/sparse.html#tensor-methods-and-sparse)

Same in tensorflow: https://www.tensorflow.org/api_docs/python/tf/sparse/SparseTensor

/azp run libertem.libertem-data

Design: could have an iterator class that is returned by `DataSet.get_partitions` which has as state the current partition size "wish". The tasks need to be submitted in batches, and based...

I've edited the issue to include a first stab at a spec for these changes - if this sounds sensible, I'll implement a prototype. /cc @uellue @matbryan52 anything to be...

> A small side quest: The tqmd progress bar can't track the number of tasks anymore, but should track processed nav size vs. total nav size. Yep, that's what I...

> I was also thinking about how to determine the starting partition size. If we start too large, we would "stare at an empty result in the GUI", and if...