Matthew Bryan
Matthew Bryan
Adds a prototype notebook for parsing a raw Timepix3 file as sparse frames. Still a WIP. Links to #1207 ## Contributor Checklist: * [x] I have added or updated my...
For datasets which define their sig shape but not their nav shape (e.g. non-integrated Merlin), if a square number of frames is detected we could guess a square scan grid...
Adds support for using sparse arrays for the `roi` parameter in `run_udf` and similar locations. Responds to #1211. Currently only supports sparse arrays from [pydata.sparse](https://sparse.pydata.org/en/stable/) as they can be arbitrary-dimensional,...
Adds an extension of `RawDataSet` which can handle groups of files and files with frame headers/footers. Responds to #1204 . Currently the implementation is not merged into `RawDataSet` or the...
Putting this up to have something to consider. ~~Still needs specific test cases, the current tests for `make_dask_array` already produced nav-aligned chunks so this is essentially a do-nothing modification for...
Looking into packaging via condaforge, it seems we're all good to go except for two things: - we depend on https://pypi.org/project/empyre/ which is also not distributed though conda-forge (or conda)....
Observed in testing #1249, a three dimensional (1-nav, 2-sig) dataset will auto-detect in the web UI as shape (N, 1) which leads to a very tall display on the page...
With the release of Scipy 1.8.0 there is an array interface for sparse matrices, allowing operations such as `array.nonzero()` and `array.count_nonzero()`, as well as slicing. (https://docs.scipy.org/doc/scipy/reference/sparse.html) This would be good...
Currently RawDataset can only handle a single raw binary file with an optional sync_offset. It shouldn't be very hard to extend it to handle lists of raw files, each with...
In checking the documentation for 0.9.0 (#1193), I realized that we haven't considered CuPy / CUDA support with the DelayedExecutor, except the convenience method `get_resources_from_udfs()` mentioned here [CUDA and Scheduling](https://libertem.github.io/LiberTEM/dask.html#cuda-and-scheduling)....