Alexander Clausen

While many observations regarding the requirements for distributed computing still stand, the performance of Spark is not satisfactory: for our use case, most of the time is spent doing copies...

There is a new interesting project: https://github.com/mars-project/mars How this relates to dask: https://github.com/mars-project/mars/issues/102 Maybe interesting to keep on our radar...

As I understand it, it doesn't work on Python functions as a basis, but on expressions on tensors. That means it can do fancy stuff like fusing expressions together (ála...

See also #706 : https://github.com/LiberTEM/LiberTEM/blob/08c93838dc5f546fcfe3b9916eeaff055ce9379e/src/libertem/io/dataset/base/backend.py#L225 :grinning: - this can be made more intelligent still, and needs to be benchmarked, but it's a start.

> About naming: ClusterDS and CacheDS to avoid confusion with dask.distributed? Currently, all `DataSet` subclasses use the full form, so that would be `ClusterDataSet` and `CacheDataSet`. As `ClusterDataSet` can also...

As a high-level API for the caching, there are at least two parts needed: 1) Settings for the cache, i.e. strategy, path, capacity, ... 2) Enabling caching for a specific...

Thanks for opening the issue! > An idea to solve the problem could be the determination of the standard deviation. Hot pixels have a very high intensity and high contrast...

The hackmd notebook referenced is [this document](https://hackmd.io/3jO3rEm4TfaPhrJm4TeZcQ). It contains notes about our recent hackathon. The idea is to use the contents of the hackmd document as a guide to improve...

> The official documentation is in github.io. Is it possible for me to change it? Sure! The documentation sources are [part of this repository](https://github.com/LiberTEM/LiberTEM/tree/master/docs/source), so you can do them like...

@uellue should we have another look at this for the 0.6 release? In #939 the docs already have a better structure, but I feel there is still quite a bit...